BenBE's humble thoughts Thoughts the world doesn't need yet …


Final crafting on GeSHi

Filed under: GeSHi — Schlagwörter: , , , , , — BenBE @ 03:24:46

After a somewhat longer period of silence from my side, due to some vacation I took, I’m proud to announce that there have quite some changes for the next release of GeSHi accumulated in the SVN trunk which will be the basis for the next release of GeSHi. As most of you might already have guessed the next release will be version, which is the first version of GeSHi in 2011 and also the first version since half a year. So what’s new with this version?

Well, the question is a bit complicated to answer, so let me split this into three parts. The first of which is all the changes to the parser of GeSHi itself. One of the changes here is a change of the handling of dashes when creating regular expressions which are used internally for GeSHi to speed up the highlighting of keywords. The problem here was that in some occasions dashes were left unescaped as part of the regexp and thus got a special meaning within character groups causing unpredictable behaviour. Although this couldn’t be used for malicious activity it was an annoying side effect causing GeSHi to crash when encountering language files which used dashes in their keywords. COBOL is one of them, Scheme another.

But let’s stay with the internal changes for another moment: There was another bug this time which affected e.g. PHP, but actually quite a bunch of other languages too. The reason for this bug is a bit more complicated to explain though, as it involves some of the internas and the precautions of GeSHi to avoid XSS attacks by the code that should be highlighted. When you look at the code sample provided there you will notice the semicolon before the offending if, right? Now, as we all know, GeSHi tries to output HTML code. This fact is important here because the semicolon – even though it doesn’t need escaping is crucial for valid HTML as it terminates escape sequences and therefore needs special treatment as we can’t simply go ahead and markup every ; we find: It might be part of an escape sequence. Luckily GeSHi works around the problem here and escapes two characters not which their HTML entity, but with something else: | with <PIPE> and – you guessed ; with <SEMI>, avoiding this disambiguity this way. Now for the problem: The default boundary checks for keywords didn’t take these replacements into account and thus hadn’t had < and > in them and therefore did NEVER match any keyword accompanied by one of those two characters. Literal < and > BTW are escaped beforehand and thus appear as &lt; and &gt; in the source when checking boundaries. Coming with this release also < and > are part of the default lists of characters allowed as boundary of a word and thus enabling the proper highlighting of the sample code in the bug report linked above.

The third issue regarding the parser is not a change of the parser itself, but rather a convenience check added to the language file checking script which didn’t verify filenames properly and thus sometimes returned invalid filenames to be checked. This bug didn’t allow for code execution, but rather produced annoying error messages when some temporary files clobbered up your language file directory.

After we’re now done with the changes to the parser let’s discuss the changes to existing language file since we have quite a few already and I’m sure I did miss even some more in the depths of my inbox! So here we go: Users of Algol68 might like the greatly improved language file by Neville Dempsey which didn’t make it into the previous release since there were some issues I needed feedback on. But even having the language file in a bit later should be early enough for you to enjoy.

Another language file which only got updated in this release is J, which is maintained by Ric Sherlock and uses one of the features fresh introduced in the previous release and now highlights all the numbers of the language J correctly, which are quite an oddity and thus needed a small adjustment of the parser to work. Or better: A tweak to make the parser look for some numebers which don’t contain digits – which actually exist in J with negative Infinity being one such example.

All friends of GDB might be in this release too, because Milian Wolff contributed an reworked and improved version for highlighting GDB stacktrace outputs making them by far more readable – believe me! So if you get the next blob of GDB output you can’t work your way through: Maybe ask GeSHi for a bit more insight. No pun intended!

Actually a bug spawning accross two language files was related to the handling of multiline comments in Javascript (and therefore also ActionScript) which both try to highlight regular expressions if they happen to detect them. The initial report for this issue was by Kevin Day who pointed me to the problem with ActionScript which could, by backporting the fix to JavaScript also be solved there. Unfortutnally I forgot one condition for JS: Multiline comments and regular expressions look SOOO close to each other, /*don’t they/ 😉 Another bug of this kind was related to F# and its prefix operators, which – you guessed it – can under certain conditions look like comments:

(*) 6 9 (* Just for reference: This comment answers questions ;-) *)

Another update to language files the new release got is related to the language GO which got some of the LangCheck warnings fixed which slipped in. Usually those don’t mean too much harm, but in regards to maintainability and consistency to what I expect from language file contributors the release should go on ahead and follow the rules, which it thanks to this patch now does.

Furtheremore in the section of updates to language files we have updates of the keyword lists for Objek (Randy Hollines), Liberty BASIC (Chris Iverson), TeraTerm (Boris Maisuradze) and Apache Configuration files (now supporting another module’s configuration options). And last but not least there are some additional comment styles for SAS (ahnolds) and fixed handling of escape sequences for CSS (yecril71pl).

And since there’s always news to report on brand new language files: here they are!

  • BASCOM AVR (Michal Goralczyk)
  • C: Loadrunner dialect (Stuart Moncrieff)
  • CoffeeScript (Trevor Burnham)
  • EPC (Thorsten Muehlfelder)
  • Falcon (billykater)
  • LLVM (Azriel Fasten)
  • UnrealScript (pospi)
  • YAML (Josh Ventura)

The two more famous of those new language probably are LLVM, an hardware-independent assembler language used as a textual representation of the intermediate code generated by the compiler framework of the same name, and YAML, which (given it’s name) ain’t markup, but serialization of data structures.

Also we have BASCOM AVR, which is used for microcontroller programming, and UnrealScript, the Scripting language used in the Unreal Engine (usually producing unreal results if you don’t know what you have to expect from your code).

So much from my side for now. Until the actual release arrives some more changes might get into, but those are definite. So look forward for the next release which will be out as soon as I manage to wrap things up.

Flattr this!


GeSHi finally packaged

Filed under: GeSHi — Schlagwörter: , , , , — BenBE @ 15:59:12

I know it’s about two months now, but I finally got around to do the last steps that were missing for GeSHi Packaging it 😉 (more…)

Flattr this!


Possible rework on the GeSHi parser waiting for review

Filed under: GeSHi — Schlagwörter: , , , — BenBE @ 12:27:04

As announced earlier here’s some external work by Simon Gábor that might get into the GeSHi core IF I can get enough confidence in those changes to work properly and don’t cause any major regressions – which I currently lack nor can confirm by reviewing this patch. (more…)

Flattr this!

GeSHi trapped in congestion of packaging pipeline

Filed under: GeSHi — Schlagwörter: , , , , , — BenBE @ 11:22:06

Hi folks,

maybe some of you already noticed, but there has already been an release of GeSHi about a month ago. If you now wonder why there is no download yet: Well, there was no time for packaging it yet, thus the only way for now is to grab the RELEASE_1_0_8_9 tag from the GeSHi SVN.

Again there are quite some changes in the release thus don’t forget to read the changelog to fetch ‚em all 😉

The next steps in getting the release out will be a small revise of a PHING release script by cweiske at the IRC channel, that aims to automate some of the stuff that has to be done for packaging, even though not necessary when just doing the tagging part of the releases. Just in case you wondered why actually packaging takes so long 😉

There will be a kinda preview of the next GeSHi release for sometime soon for all the changes that already got integrated so far. Mostly this affects a lot of new language files and changes to existing ones but minor patches to the LangCheck script are waiting too. To grab it just have a look at the RELEASE_STABLE branch at the GeSHi SVN; updates will go there. Currently I’m still trying to reduce the backlog of mails that accumulated, But I’m mostly done with it.

There’s one somewhat bigger patch still waiting in the pipeline that addresses an issue with „highlight-within-highlighted“ situations but due to major changes to the parser this patch introduces I didn’t review it yet. But more on this in another post.

Flattr this!


A brief overview on Syntax Highlighting

Filed under: GeSHi — Schlagwörter: , , , , — BenBE @ 19:17:12

The following is part of a weekly column by Ido Gendel for the Israeli online magazine, which he allowed me to publish in its English translation in my blog. Besides a short history of syntax highlighting it also covers a short interview which we made by mail. Happy reading! (more…)

Flattr this!


Regular Expression Highlighting

Filed under: GeSHi — Schlagwörter: , , , — BenBE @ 17:43:15

I just found some feature request asking nicely for some regular expression highlighting support in GeSHi. And since I can’t deny any requests that are by nature to lead to obscure code: Here it is 😉 (more…)

Flattr this!


GeSHi is on its way

Filed under: GeSHi — Schlagwörter: , , — BenBE @ 13:19:46

Many people might have noticed, and even though I didn’t do that many public announcements of this yet, GeSHi is still under developement and a new version is coming near its completion soon – hopefully. The reason I mention this is, that some distributions – like Debian – haven’t updated to the available packages yet even though (at least in the case of Debian) I notified them of at least one intermediate release. Well, probably they only look onto pretty homepages to determine the news instead of watching the code. So well, here’s some news for you all. (more…)

Flattr this!


Cross-Domain XmlHttpRequest-Foo mit Custom HTTP Methods

Filed under: Software — Schlagwörter: , , , , , , , , , , — BenBE @ 04:36:32

Kleine Vorwarnung vorweg: Es wird technisch! Wem die Überschrift nichts sagt, empfehle ich vor dem Fortsetzen in diesem Post das Studium diverser Standards. Zudem sei bereits hier angemerkt: Die hier vorgestellte Technik funktioniert auf Grund der Same Origin Policy nicht mit allen Browsern. Getestet (und als Funktionierend bekannt) ist aber mindestens alles, was FF 3.5.X oder besser heißt. Aber genug der Vorrede, fangen wir an. (more…)

Flattr this!


Styling GeSHi output

Filed under: GeSHi — Schlagwörter: , , — BenBE @ 19:17:53

When highlighting source for your website you often want to make it use a common style accross languages or want to provide multiple themes for one language. In this short howto I want to give some details on how you can actually do this. (more…)

Flattr this!


Some updates on developement

Filed under: GeSHi — Schlagwörter: , , , — BenBE @ 23:13:00

Many might have wondered why there hasn’t been any official release for quite some time and well, that’s a somewhat complicated story to tell. But let’s start with the easy parts. (more…)

Flattr this!

« Newer PostsOlder Posts »

Powered by WordPress