6. The method of claim 1 , further comprising repeating steps (b)-(e).
1. A method for providing a training set to build a statistical relevancy scoring function for a document relative to selected terms in a lexicon, comprising: in a search engine that accesses servers of documents in a computer network, (a) identifying an initial set of hypertext documents in a collection of documents as a training set of relevant documents; (b) identifying hyperlinks included in each hypertext document of the training set; (c) including in the training set the hypertext documents pointed to by the identified hyperlinks; (d) identifying anchortexts associated with the hypertext documents of the training set; and (e) including the anchortexts in the lexicon; wherein the statistical scoring function is determined by combining individual contributions to the statistical scoring function by each of the selected terms, wherein the individual contribution by each selected term is related to a term frequency, being the frequency of occurrence of that selected term in the document, and a document frequency, being the number of documents in the collection of documents that include that selected term.