1. A computer-implemented method comprising: extracting, by a processor, text from a document; identifying, by the processor, terms within the extracted text, each of the terms comprising a contiguous grouping of two or more tokens, each token comprising a word; determining, by the processor, a token value representing a total number of times each token occurs in the document; determining, by the processor, a token frequency for each of the terms as a function of the token values of tokens in each of the terms; and ranking, by the processor, the terms using the token frequency determined for each of the terms.
1. A computer-implemented method comprising: extracting, by a processor, text from a document; identifying, by the processor, terms within the extracted text, each of the terms comprising a contiguous grouping of two or more tokens, each token comprising a word; determining, by the processor, a token value representing a total number of times each token occurs in the document; determining, by the processor, a token frequency for each of the terms as a function of the token values of tokens in each of the terms; and ranking, by the processor, the terms using the token frequency determined for each of the terms. 3. The method of claim 1 further comprising determining a term value for each of the terms representing a total number of times each of the terms occurs in document, wherein ranking the terms further comprises using the term value for each of the terms.