6. The improvement of claim 2 where selecting high quality grams comprises: if a gram g has a low frequency, eliminating from the frequency trie all the extended grams of g; and if a gram is very frequent, keeping selected ones of the corresponding extended grams in the frequency trie.
6. The improvement of claim 2 where selecting high quality grams comprises: if a gram g has a low frequency, eliminating from the frequency trie all the extended grams of g; and if a gram is very frequent, keeping selected ones of the corresponding extended grams in the frequency trie. 8. The improvement of claim 6 where if a gram g has a low frequency, eliminating from the frequency trie all the extended grams of g comprises: choosing a frequency threshold, T; and pruning the frequency trie by checking nodes from the root down to determine if a current node n has a leaf-node child marked by an edge labeled by the end marker symbol #, if the current node n does not have any leaf-node child, then the path from the root to the current node n corresponds to a gram shorter than q min , thus recursively pruning the frequency trie for each of the current node n's children, if the current node n node has a leaf-node child L, then materializing a gram g corresponding to L with the frequency of node n, n.freq, if the frequency n.freq is already not greater than T, then we keeping the gram corresponding to leaf-node child L in the frequency trie, and then removing the children of current node n except for leaf-node child L, and assigning a frequency of n to leaf-node child L, so that after this pruning step, current node n has a single leaf-node child L.