4. A system comprising: a computer-readable storage medium including instructions; and data processing apparatus capable of executing the instructions to perform operations comprising: receiving a collection of n-grams, each n-gram having a corresponding first probability of occurring; generating a trie using a first vector and a second vector, the trie assigning one or more values identifying each n-gram in the collection of n-grams, and in which the first vector is a left word vector identifying each distinct left word for a given right context in the collection, the second vector is a diversity count vector identifying a count of distinct left words for each right context in the collection; and using the trie to identify the probabilities of occurring for corresponding n-grams in a received string that includes one or more n-grams.
8. The system of claim 4 in which generating the trie further comprises using a third vector and a fourth vector, where the first vector identifies each distinct left word for each distinct context in the collection, the second vector identifies a count of distinct left words for each context in the collection, the third vector identifies each distinct predicted word for a given context in the collection, and the fourth vector identifies a count of distinct predicted words for each context in the collection.