1. A method for crawling a web site, the method comprising the steps of: a) querying a web site server by a crawler program, wherein at least one page of the web site has a reference for executing by a browser to produce an address for a next page; b) parsing such a reference from one of the web pages by the crawler program and sending the reference to an applet running in the browser; and c) determining the address for the next page by the browser responsive to the reference and sending the address to the crawler.