1. A computer-based method of transforming a natural language query into a representation of the natural language query wherein the representation is usable for purposes of an input into a search system that extracts answers based on the input, the computer-based method comprising: utilizing a computer processor for comparing the natural language query against common terms of a core knowledge pack to identifying semantic units in the natural language query such that each semantic unit is a respective portion of the natural language query; utilizing the computer processor for associating a token with each uniquely identified semantic unit by recognizing the respective uniquely identified semantic unit in a dictionary having the token associated with the uniquely identified semantic unit; utilizing the computer processor for identifying a stem for at least a first one of the tokens as part of a token processing operation, the stem being identified by replacing the first token with a stem corresponding to the token in the dictionary wherein the token associated with the stem is also associated with a plurality of semantic units in the dictionary; utilizing the computer processor for identifying a lexical phrase for at least a second one of the tokens as part of the token processing operation, wherein the lexical phrase is obtained by combining one of the uniquely identified semantic units with the second token; and utilizing the computer processor for representing the query as an ordered combination of the identified stems and lexical phrases identified in the token processing operation.