1. An ecommerce system having a database comprising multiple data items, wherein the data items represent products sold in the ecommerce system, the system comprising: a processor; a processor-implemented candidate list generator module to: generate a list of keywords from search query information; generate a set of token pairs including a keyword from the list of keywords and a token, the token being a synonym of the keyword; receive demand information and supply information, wherein the demand information is information retrieved from query logs maintained for user-provided query entries stored in the database and the supply information is seller-provided descriptor information for the data items stored in the database; and apply candidate selection rules to token pairs across a plurality of categories of data items in the database using the demand information and the supply information, generating a set of candidate token pairs; a processor-implemented validation module to: calculate a divergence value for each candidate token pair across the plurality of categories, wherein calculation of the divergence value comprises: performing a search using a keyword of a candidate token pair as a search criteria; determining a first distribution (P) of search results for the keyword, the first distribution over the plurality of categories; performing a search using a token of the candidate token pair as a search criteria; determining a second distribution (Q) of search results for the token, the second distribution over the plurality of categories; calculating the divergence measure as an Information Radius (IRad) value as: 
 IRad=0.5[ D ( P ∥avg( P,Q ))+ D ( Q ∥avg( P,Q ))], wherein D is Kullback-Liebler divergence calculated as: 
 D ( P∥Q )=Σ P ( i )log( P ( i )/ Q ( i )), wherein avg (P,Q) is an average of the first distribution and the second distribution, and wherein an index i=1, 2, . . . N, and N is the number of categories in the plurality of categories; and validate the candidate token pairs based on the divergence value, wherein the candidate token pairs having divergence values exceeding a divergence threshold are valid token pairs; and a processor-implemented data dictionary module to: receive the validated token pairs as entries in a vocabulary, and provide the validated token pairs in response to a search query containing a keyword corresponding to at least one entry in the vocabulary.
1. An ecommerce system having a database comprising multiple data items, wherein the data items represent products sold in the ecommerce system, the system comprising: a processor; a processor-implemented candidate list generator module to: generate a list of keywords from search query information; generate a set of token pairs including a keyword from the list of keywords and a token, the token being a synonym of the keyword; receive demand information and supply information, wherein the demand information is information retrieved from query logs maintained for user-provided query entries stored in the database and the supply information is seller-provided descriptor information for the data items stored in the database; and apply candidate selection rules to token pairs across a plurality of categories of data items in the database using the demand information and the supply information, generating a set of candidate token pairs; a processor-implemented validation module to: calculate a divergence value for each candidate token pair across the plurality of categories, wherein calculation of the divergence value comprises: performing a search using a keyword of a candidate token pair as a search criteria; determining a first distribution (P) of search results for the keyword, the first distribution over the plurality of categories; performing a search using a token of the candidate token pair as a search criteria; determining a second distribution (Q) of search results for the token, the second distribution over the plurality of categories; calculating the divergence measure as an Information Radius (IRad) value as: 
 IRad=0.5[ D ( P ∥avg( P,Q ))+ D ( Q ∥avg( P,Q ))], wherein D is Kullback-Liebler divergence calculated as: 
 D ( P∥Q )=Σ P ( i )log( P ( i )/ Q ( i )), wherein avg (P,Q) is an average of the first distribution and the second distribution, and wherein an index i=1, 2, . . . N, and N is the number of categories in the plurality of categories; and validate the candidate token pairs based on the divergence value, wherein the candidate token pairs having divergence values exceeding a divergence threshold are valid token pairs; and a processor-implemented data dictionary module to: receive the validated token pairs as entries in a vocabulary, and provide the validated token pairs in response to a search query containing a keyword corresponding to at least one entry in the vocabulary. 2. The system of claim 1 , wherein the candidate list generator module comprises candidate selection rules, including vocabulary generation rules for processing search query terms to identify keywords, and token generation rules for generating synonyms of the keywords.