BenBE's humble thoughts Thoughts the world doesn't need yet …

After a somewhat longer period of silence from my side, due to some vacation I took, I’m proud to announce that there have quite some changes for the next release of GeSHi accumulated in the SVN trunk which will be the basis for the next release of GeSHi. As most of you might already have guessed the next release will be version 1.0.8.10, which is the first version of GeSHi in 2011 and also the first version since half a year. So what’s new with this version?

Well, the question is a bit complicated to answer, so let me split this into three parts. The first of which is all the changes to the parser of GeSHi itself. One of the changes here is a change of the handling of dashes when creating regular expressions which are used internally for GeSHi to speed up the highlighting of keywords. The problem here was that in some occasions dashes were left unescaped as part of the regexp and thus got a special meaning within character groups causing unpredictable behaviour. Although this couldn’t be used for malicious activity it was an annoying side effect causing GeSHi to crash when encountering language files which used dashes in their keywords. COBOL is one of them, Scheme another.

But let’s stay with the internal changes for another moment: There was another bug this time which affected e.g. PHP, but actually quite a bunch of other languages too. The reason for this bug is a bit more complicated to explain though, as it involves some of the internas and the precautions of GeSHi to avoid XSS attacks by the code that should be highlighted. When you look at the code sample provided there you will notice the semicolon before the offending if, right? Now, as we all know, GeSHi tries to output HTML code. This fact is important here because the semicolon – even though it doesn’t need escaping is crucial for valid HTML as it terminates escape sequences and therefore needs special treatment as we can’t simply go ahead and markup every ; we find: It might be part of an escape sequence. Luckily GeSHi works around the problem here and escapes two characters not which their HTML entity, but with something else: | with <PIPE> and – you guessed ; with <SEMI>, avoiding this disambiguity this way. Now for the problem: The default boundary checks for keywords didn’t take these replacements into account and thus hadn’t had < and > in them and therefore did NEVER match any keyword accompanied by one of those two characters. Literal < and > BTW are escaped beforehand and thus appear as &lt; and &gt; in the source when checking boundaries. Coming with this release also < and > are part of the default lists of characters allowed as boundary of a word and thus enabling the proper highlighting of the sample code in the bug report linked above.

The third issue regarding the parser is not a change of the parser itself, but rather a convenience check added to the language file checking script which didn’t verify filenames properly and thus sometimes returned invalid filenames to be checked. This bug didn’t allow for code execution, but rather produced annoying error messages when some temporary files clobbered up your language file directory.

After we’re now done with the changes to the parser let’s discuss the changes to existing language file since we have quite a few already and I’m sure I did miss even some more in the depths of my inbox! So here we go: Users of Algol68 might like the greatly improved language file by Neville Dempsey which didn’t make it into the previous release since there were some issues I needed feedback on. But even having the language file in a bit later should be early enough for you to enjoy.

Another language file which only got updated in this release is J, which is maintained by Ric Sherlock and uses one of the features fresh introduced in the previous release and now highlights all the numbers of the language J correctly, which are quite an oddity and thus needed a small adjustment of the parser to work. Or better: A tweak to make the parser look for some numebers which don’t contain digits – which actually exist in J with negative Infinity being one such example.

All friends of GDB might be in this release too, because Milian Wolff contributed an reworked and improved version for highlighting GDB stacktrace outputs making them by far more readable – believe me! So if you get the next blob of GDB output you can’t work your way through: Maybe ask GeSHi for a bit more insight. No pun intended!

Actually a bug spawning accross two language files was related to the handling of multiline comments in Javascript (and therefore also ActionScript) which both try to highlight regular expressions if they happen to detect them. The initial report for this issue was by Kevin Day who pointed me to the problem with ActionScript which could, by backporting the fix to JavaScript also be solved there. Unfortutnally I forgot one condition for JS: Multiline comments and regular expressions look SOOO close to each other, /*don’t they/ 😉 Another bug of this kind was related to F# and its prefix operators, which – you guessed it – can under certain conditions look like comments:

(*) 6 9 (* Just for reference: This comment answers questions ;-) *)

Another update to language files the new release got is related to the language GO which got some of the LangCheck warnings fixed which slipped in. Usually those don’t mean too much harm, but in regards to maintainability and consistency to what I expect from language file contributors the release should go on ahead and follow the rules, which it thanks to this patch now does.

Furtheremore in the section of updates to language files we have updates of the keyword lists for Objek (Randy Hollines), Liberty BASIC (Chris Iverson), TeraTerm (Boris Maisuradze) and Apache Configuration files (now supporting another module’s configuration options). And last but not least there are some additional comment styles for SAS (ahnolds) and fixed handling of escape sequences for CSS (yecril71pl).

And since there’s always news to report on brand new language files: here they are!

• BASCOM AVR (Michal Goralczyk)
• C: Loadrunner dialect (Stuart Moncrieff)
• CoffeeScript (Trevor Burnham)
• EPC (Thorsten Muehlfelder)
• Falcon (billykater)
• LLVM (Azriel Fasten)
• UnrealScript (pospi)
• YAML (Josh Ventura)

The two more famous of those new language probably are LLVM, an hardware-independent assembler language used as a textual representation of the intermediate code generated by the compiler framework of the same name, and YAML, which (given it’s name) ain’t markup, but serialization of data structures.

Also we have BASCOM AVR, which is used for microcontroller programming, and UnrealScript, the Scripting language used in the Unreal Engine (usually producing unreal results if you don’t know what you have to expect from your code).

So much from my side for now. Until the actual release arrives some more changes might get into, but those are definite. So look forward for the next release which will be out as soon as I manage to wrap things up.