2. The method of claim 1 , wherein determining whether the URL is a root URL or a leaf URL comprises determining that the URL is a root URL if the URL does not include a directory path.
1. A method comprising: maintaining a URL index comprising a plurality of URLs; parsing, using at least one processor and without user intervention, a URL from the plurality of URLs to identify a domain name associated with the URL; comparing a directory path associated with the URL to a list of ISP client directory paths to determine if the directory path matches an ISP client directory path included in the list of ISP client directory paths; determining whether the URL is a root URL or a leaf URL; updating the URL index to reflect the determination of whether the URL is a root URL or a leaf URL; facilitating one or more queries of the URL index; and providing, in response to the one or more queries, one or more results that reflect the determination of whether the URL is a root URL or a leaf URL.