15. The method of claim 12 , further comprising: determining which of the one or more category clusters to promote into a primary category cluster or a secondary category cluster, or to demote.
12. A method of optimized web domains classification based on progressive crawling with clustering, comprising: crawling a domain to collect data for a subset of pages of a corpus of content associated with the domain; classifying each of the crawled pages into one or more category clusters, wherein the category clusters represent a content categorization of the corpus of content associated with the domain, and wherein the classifying of the each of the crawled pages into the one or more category clusters comprises: determining a category for the each of the crawled pages in the domain; grouping more than one page having the same category into a first cluster; determining whether a number of the more than one page of the first cluster exceeds a first threshold; and in the event that the number of the more than one page of the first cluster does not exceed the first threshold, selecting a new page within the domain to crawl and classify; and determining which of the one or more category clusters to publish for the domain.