17. A method comprising: maintaining a URL database comprising one or more URLs; retrieving one or more documents associated with the one or more URLs; parsing, using at least one processor, the one or more documents to identify a first set of new URLs within the one or more documents; determining if one or more URLs in the first set of URLs are root URLs based on a set of predefined rules; if it is determined that one or more URLs in the first set of URLs are root URLs, classifying the one or more URLs in the first set of URLs as root URLs in the URL database and updating the URL database to include the identified first set of new URLs including the one or more URLs classified as root URLs; querying, without user intervention, a third-party search engine using one or more keywords to identify a second set of new URLs; and updating the URL database to include the identified second set of new URLs.
17. A method comprising: maintaining a URL database comprising one or more URLs; retrieving one or more documents associated with the one or more URLs; parsing, using at least one processor, the one or more documents to identify a first set of new URLs within the one or more documents; determining if one or more URLs in the first set of URLs are root URLs based on a set of predefined rules; if it is determined that one or more URLs in the first set of URLs are root URLs, classifying the one or more URLs in the first set of URLs as root URLs in the URL database and updating the URL database to include the identified first set of new URLs including the one or more URLs classified as root URLs; querying, without user intervention, a third-party search engine using one or more keywords to identify a second set of new URLs; and updating the URL database to include the identified second set of new URLs. 18. The method of claim 17 , further comprising obtaining geographical information associated with the first URL from the third-party database.