16. The system of claim 15 , wherein the instructions cause the one or more computers to perform operations further comprising: selecting, from the set of images, a plurality of first image triplets that each include a triplet reference image and two unlabeled images; determining, for each first image triplet, a distance between feature vectors of the unlabeled images and a feature vector for the triplet reference image; and selecting subsets of unlabeled images based, at least in part, on the distance for each first image triplet.
15. A system comprising: a data storage device storing, for a set of images, image feature data specifying visual features of images in the set of images; and one or more computers coupled to the data storage device, the data processing apparatus including instructions that cause the data processing apparatus to perform operations comprising: ranking, based on a first measure of visual similarity of each unlabeled image to a first reference image, unlabeled images in a set of images that includes at least two unlabeled images; generating a weighted feature vector based on visual features of the unlabeled images and the ranking; identifying, from the set of images, a subset of labeled images that includes a second reference image and at least two labeled images; determining, based on the visual features of the at least two labeled images and the weighted feature vector, a second measure of visual similarity between the second reference image and each of the at least two labeled images, the second measure of visual similarity being determined independent of similarity feedback data indicative of user specified visual similarity between the second reference image and each of the at least two labeled images; ranking the at least two labeled images based on the second measure of visual similarity associated with each of the labeled images; determining that the ranking of the labeled images does not match a second ranking of the labeled images based on the similarity feedback data; and adjusting the weighted feature vector in response to the determination that the ranking does not match the second ranking and based on the similarity feedback data.