1. A method performed by data processing apparatus having memory, a processor, and code stored in the memory and executed in the processor, the method comprising: receiving by the processor training data that specify manual classifications of content items and feature values for each of the content items, the manual classification for each of the content items specifying whether the content item is a violating content item that violates content item distribution guidelines, the feature values specifying one or more characteristics of the content items and characteristics of landing pages to which the content items link; training by the processor a compliance model using the training data, the compliance model being trained to classify an unclassified content item as a violating content item based on the feature values of the unclassified content item; determining by the processor that the compliance model has an accuracy measure that meets a threshold accuracy measure; in response to determining that the accuracy measure for the compliance model meets the accuracy threshold, classifying by the processor unclassified content items using the feature values for the unclassified content items; and providing by the processor data specifying the classifications of the unclassified content items; wherein classifying unclassified content items comprises classifying at least one unclassified content item as a suspicious content item, and further comprising: providing the suspicious content item to a rater for manual classification; receiving data specifying a manual classification of the suspicious content item; and updating the compliance model using the manual classification and the feature values of the suspicious content item.