5. The method of claim 4 , wherein the step of estimating the impact of fetching the particular uncrawled web page further comprises combining a query-based estimate and a query-independent estimate for said particular uncrawled web page by computing a weighted average of said query-based estimate and said query-independent estimate for said particular uncrawled web page; wherein said query-based estimate for said particular uncrawled web page indicates how well said particular uncrawled web page meets neediness and relevance requirements of the needy queries and is determined based, at least in part, on the anchortext and the URL that point to the particular uncrawled web page; wherein said query-independent estimate of said particular uncrawled web page is determined based on query-independent features of said particular uncrawled web page.
6. The method of claim 5 , wherein the step of estimating the impact of fetching the particular uncrawled web page further comprises creating query sketches, for said needy queries, by computing an expected impact score for the particular uncrawled web page and expected impact scores for the previously crawled web pages that match said needy queries, and uses the query sketches to determine the query-based estimate and the query-independent estimate for the particular uncrawled web page.