20. The computer system of claim 19 , wherein the instructions for storing crawl data further comprises instructions for determining an average quantity of time expended to access the documents from the respective website during the one or more crawl sessions.
19. The computer system of claim 13 , wherein the instructions for storing crawl data further comprises instructions for determining a number documents of the respective website that are accessed during one or more crawl sessions.