1. A method of building an index of web pages, the method comprising: accessing a set of URLs collected by crawling the Internet; accessing a list of URLs collected from one or more sources that collect clicks of URLs by users; for each URL in the set of URLs, for a given URL: computing a measure of likelihood that the given URL will be searched by a user in the future based on whether the URL has been clicked by a user, storing the given URL and its measure, selecting a subset of the URLs based on their respective stored measures, where some of the URLs in the set of URLs are omitted from the subset based on the measures of the omitted URLs; generating a first index of the web pages pointed to by the URLs in the subset of URLs, the first index not including the omitted URLs, the first index comprising a mapping between contents of the web pages and the URLs of the web pages, and generating a second index of the web pages pointed to by the URLs omitted from the subset, the second index not including the URLs in the first index; and using the index by a search engine to search for search results for arbitrary search queries submitted by users, wherein when the first index does not satisfy a given query, using the second index to attempt to satisfy the given query.
1. A method of building an index of web pages, the method comprising: accessing a set of URLs collected by crawling the Internet; accessing a list of URLs collected from one or more sources that collect clicks of URLs by users; for each URL in the set of URLs, for a given URL: computing a measure of likelihood that the given URL will be searched by a user in the future based on whether the URL has been clicked by a user, storing the given URL and its measure, selecting a subset of the URLs based on their respective stored measures, where some of the URLs in the set of URLs are omitted from the subset based on the measures of the omitted URLs; generating a first index of the web pages pointed to by the URLs in the subset of URLs, the first index not including the omitted URLs, the first index comprising a mapping between contents of the web pages and the URLs of the web pages, and generating a second index of the web pages pointed to by the URLs omitted from the subset, the second index not including the URLs in the first index; and using the index by a search engine to search for search results for arbitrary search queries submitted by users, wherein when the first index does not satisfy a given query, using the second index to attempt to satisfy the given query. 5. A method according to claim 1 , wherein the determining whether the given URL has been clicked by a user is based on click data collected by web browsers, the click data comprising URLs captured by browsers when users clicked on URLs in web pages displayed in the browsers.