15. The method of claim 1 , wherein the second predetermined threshold is about 100.
1. A computer implemented method for identifying phrases in a document collection, the method comprising: collecting possible phrases from documents in the document collection; classifying individual possible phrases as either a good phrase or a bad phrase according to a frequency of occurrence of the individual possible phrase; determining, for a pair of good phrases g j and g k in the document collection, an information gain of g k with respect to g j as a function of an actual co-occurrence rate of g j and g k and an expected co-occurrence rate of g j and g k in the document collection; selectively retaining as valid phrases those good phrases that, predict the occurrence of at least one other good phrase in the document collection, where a good phrase g j predicts the occurrence of another good phrase g k in the document collection when the determined information gain of g k in the presence of g j exceeds a first predetermined threshold; identifying, for a plurality of selectively retained valid phrases g x , a phrase g y as a related phrase of g x where the information gain of g y in the presence of g x exceeds a second predetermined threshold that is more restrictive than the first predetermined threshold; and storing the valid phrases and identified related phrases on a computer-readable storage medium.