14. A device comprising: a memory to store instructions; and a processor to execute the instructions to implement: a receiving component to receive a sequence of terms; a coherence component to calculate a coherence of multiple terms in the sequence of terms; a variation component to calculate a variation of context terms in a collection of documents in which the sequence occurs, where the variation of context terms is calculated as a measure of entropy of the context of the sequence; and a decision component to determine whether the sequence constitutes a semantic unit based at least in part on results of the coherence component and the variation component, and output an indication of whether the sequence constitutes a semantic unit for use in a processor.
14. A device comprising: a memory to store instructions; and a processor to execute the instructions to implement: a receiving component to receive a sequence of terms; a coherence component to calculate a coherence of multiple terms in the sequence of terms; a variation component to calculate a variation of context terms in a collection of documents in which the sequence occurs, where the variation of context terms is calculated as a measure of entropy of the context of the sequence; and a decision component to determine whether the sequence constitutes a semantic unit based at least in part on results of the coherence component and the variation component, and output an indication of whether the sequence constitutes a semantic unit for use in a processor. 23. The device of claim 14 , where the processor further executes the instructions to implement: a heuristics component to apply one or more predefined rules to the sequence, where the decision component is further to determine whether the sequence constitutes a semantic unit based at least in part on application of the one or more rules.