16. The system of claim 12 , wherein the one or more computers are further operable to perform operations including: determining an updated accuracy measure for the updated compliance model; determining that the updated accuracy measure does not meet the accuracy measure for the compliance model; and in response to determining that the updated accuracy measure does not meet the accuracy measure, classifying content items using the compliance model.
12. A system comprising: a data store storing training data that specify manual classifications of content items and feature values for each of the content items, the manual classification for each of the content items specifying whether the content item is a violating content item that violates content item distribution guidelines, the feature values specifying one or more characteristics of the content items and characteristics of landing pages to which the content items link; and one or more computers, each having a memory, a processor, and code stored in the memory and executable in the processor, the one or more computers being operable to interact with the data store and to cause the processor to perform operations including: receiving the training data; training a compliance model using the training data, the compliance model being trained to classify an unclassified content item as a violating content item based on the feature values of the unclassified content item; determining that the compliance model has an accuracy measure that meets a threshold accuracy measure; in response to determining that the accuracy measure for the compliance model meets the accuracy threshold, classifying unclassified content items using the feature values for the unclassified content items; and providing data specifying the classifications of the unclassified content items; wherein the one or more computers are further operable to perform operations including: classifying at least one unclassified content item as a suspicious content item, and further comprising: providing the suspicious content item to a rater for manual classification; receiving data specifying a manual classification of the suspicious content item; and updating the compliance model using the manual classification and the feature values of the suspicious content item.