Description: ------------ I noticed that the regular expressions used in the lexer were constructed in a manner of preg_match() on a substr() from a certain offset of the text. A better performing solution is to use the 'offset' parameter of preg_match(), in combination with the \G offset place identifier in the regular expression. The performance improvement is quite significant (not orders of magnitude), and it probably also reduces alot of memory swapping/usage due to the string not being chopped up at every match.
Changes integrated with other work; update is in CVS.