1. In a computer system, a method for processing data stored as input strings, the input strings including a plurality of sub-strings, each sub-string associated with a value, the values of the sub-strings distributed over an interval, the interval of the values including a plurality of ranges of the values, comprising arranging the sub-strings into disjoint sets, each set including the sub-strings having values selected from a predetermined range of the interval of the values, a prefix of the sub-strings of each set being identical; and for each input string repeatedly truncating the input string, by deleting the prefix of a leading sub-string of the input string to produce a token in an output string, until the input strings have been parsed into the tokens of the output strings.