1. A method implemented by a data processing apparatus, comprising: receiving, for each image in a set of images, a corresponding set of labels, each label being determined to be indicative of subject matter of the image; for each label, determining, by the data processing apparatus, one or more confidence values, each confidence value being a measure of confidence that the label accurately describes the subject matter of a threshold number of respective images to which it corresponds; identifying, by the data processing apparatus, labels for which each of the one or more confidence values meets a respective confidence threshold as high confidence labels; for each image in the set of images, identifying, by the data processing apparatus, the labels in its corresponding set of labels that are high confidence labels; and identifying, by the data processing apparatus, images having a corresponding set of labels that include at least a respective threshold number of high confidence labels as high confidence images.