1. An ecommerce system having a database comprising multiple data items, wherein the data items represent products sold in the ecommerce system, the system comprising: a processor; a processor-implemented candidate list generator module to: generate a list of keywords from search query information; generate a set of token pairs including a keyword from the list of keywords and a token, the token being a synonym of the keyword; receive demand information and supply information, wherein the demand information is information retrieved from query logs maintained for user-provided query entries stored in the database and the supply information is seller-provided descriptor information for the data items stored in the database; and apply candidate selection rules to token pairs across a plurality of categories of data items in the database using the demand information and the supply information, generating a set of candidate token pairs; a processor-implemented validation module to: calculate a divergence value for each candidate token pair across the plurality of categories, wherein calculation of the divergence value comprises: performing a search using a keyword of a candidate token pair as a search criteria; determining a first distribution (P) of search results for the keyword, the first distribution over the plurality of categories; performing a search using a token of the candidate token pair as a search criteria; determining a second distribution (Q) of search results for the token, the second distribution over the plurality of categories; calculating the divergence measure as an Information Radius (IRad) value as: 
 IRad=0.5[ D ( P ∥avg( P,Q ))+ D ( Q ∥avg( P,Q ))], wherein D is Kullback-Liebler divergence calculated as: 
 D ( P∥Q )=Σ P ( i )log( P ( i )/ Q ( i )), wherein avg (P,Q) is an average of the first distribution and the second distribution, and wherein an index i=1, 2, . . . N, and N is the number of categories in the plurality of categories; and validate the candidate token pairs based on the divergence value, wherein the candidate token pairs having divergence values exceeding a divergence threshold are valid token pairs; and a processor-implemented data dictionary module to: receive the validated token pairs as entries in a vocabulary, and provide the validated token pairs in response to a search query containing a keyword corresponding to at least one entry in the vocabulary.
1. An ecommerce system having a database comprising multiple data items, wherein the data items represent products sold in the ecommerce system, the system comprising: a processor; a processor-implemented candidate list generator module to: generate a list of keywords from search query information; generate a set of token pairs including a keyword from the list of keywords and a token, the token being a synonym of the keyword; receive demand information and supply information, wherein the demand information is information retrieved from query logs maintained for user-provided query entries stored in the database and the supply information is seller-provided descriptor information for the data items stored in the database; and apply candidate selection rules to token pairs across a plurality of categories of data items in the database using the demand information and the supply information, generating a set of candidate token pairs; a processor-implemented validation module to: calculate a divergence value for each candidate token pair across the plurality of categories, wherein calculation of the divergence value comprises: performing a search using a keyword of a candidate token pair as a search criteria; determining a first distribution (P) of search results for the keyword, the first distribution over the plurality of categories; performing a search using a token of the candidate token pair as a search criteria; determining a second distribution (Q) of search results for the token, the second distribution over the plurality of categories; calculating the divergence measure as an Information Radius (IRad) value as: 
 IRad=0.5[ D ( P ∥avg( P,Q ))+ D ( Q ∥avg( P,Q ))], wherein D is Kullback-Liebler divergence calculated as: 
 D ( P∥Q )=Σ P ( i )log( P ( i )/ Q ( i )), wherein avg (P,Q) is an average of the first distribution and the second distribution, and wherein an index i=1, 2, . . . N, and N is the number of categories in the plurality of categories; and validate the candidate token pairs based on the divergence value, wherein the candidate token pairs having divergence values exceeding a divergence threshold are valid token pairs; and a processor-implemented data dictionary module to: receive the validated token pairs as entries in a vocabulary, and provide the validated token pairs in response to a search query containing a keyword corresponding to at least one entry in the vocabulary. 7. The system of claim 1 , wherein the vocabulary is a stemming and transliteration vocabulary.