10. A non-transitory computer-readable medium storing one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of: receiving, at a search engine operating on one or more computing devices from a traffic server that is not the search engine and is not operating on the one or more computing devices, network usage information that indicates a frequency with which users of a plurality of users requested, during a particular period of time, a particular document through the traffic server, which monitors frequencies with which documents in a plurality of documents are provided, through the traffic server, from a plurality of origin servers upon which the plurality of documents reside, to the plurality of users in response to a plurality of document-identifying requests from the plurality of users; positioning the particular document within a queue at a queue position that is selected based on said network usage information; performing a crawling or indexing operation based at least in part on the position of the particular document within the queue.
10. A non-transitory computer-readable medium storing one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of: receiving, at a search engine operating on one or more computing devices from a traffic server that is not the search engine and is not operating on the one or more computing devices, network usage information that indicates a frequency with which users of a plurality of users requested, during a particular period of time, a particular document through the traffic server, which monitors frequencies with which documents in a plurality of documents are provided, through the traffic server, from a plurality of origin servers upon which the plurality of documents reside, to the plurality of users in response to a plurality of document-identifying requests from the plurality of users; positioning the particular document within a queue at a queue position that is selected based on said network usage information; performing a crawling or indexing operation based at least in part on the position of the particular document within the queue. 15. The computer-readable medium of claim 10 , wherein the one or more sequences of instructions, when executed by the one or more processors, further cause the one or more processors to perform monitoring said frequency that said particular document is requested by the users of the plurality of users.