9. The system recited in claim 1 , the processor is further configured to: determine whether to promote or demote a cluster as a category cluster.
1. A system for providing web domains classification based on progressive crawling with clustering, comprising: a processor configured to: distribute a first Uniform Resource Locator (URL) content categorization data feed to a first plurality of subscribers, comprising to: collect the first URL content categorization data feed based on progressive crawling with clustering to determine which category clusters to publish for each categorized web domain, comprising to: automatically classify content for a first web domain, comprising to: receive the first web domain from a first security device associated with one of the first plurality of subscribers, comprising to: crawl a plurality of pages in the first web domain; and determine a category for the plurality of pages in the first web domain, to group more than one page having the same category into a first cluster, comprising to: determine whether a number of the more than one page of the first cluster exceeds a first threshold; in the event that the number of the more than one page of the first cluster does not exceed the first threshold, select a new page within the web domain to crawl; and classify the new page; and send a classification determined for the content for the first web domain to the first security device associated with one of the first plurality of subscribers; and a memory coupled to the processor and configured to provide the processor with instructions.