7. The method of claim 1 , wherein the maximum number of dictionary entries is configured such that each definition of the plurality of entries in the static dictionary can fit within a processor cache of the one or more computing devices.
1. A method comprising: determining a static dictionary from a portion of input data, the static dictionary comprising a plurality of entries up to a maximum number of dictionary entries, each of the plurality of entries mapping a token to a definition having a length up to a maximum byte size that is bounded by a hardware specification; tokenizing the input data using the static dictionary to generate a packed sequential plurality of tokens, each of the packed sequential plurality of tokens having a fixed token size that is configured to address the maximum number of dictionary entries; storing the static dictionary and the packed sequential plurality of tokens; wherein the method is performed by one or more computing devices.