17. A method performed by a data processing apparatus, comprising: storing data identifying a plurality of training images for a query, wherein each of the training images is classified as being in a positive group of images for the query or a negative group of images for the query according to a respective query specific preference measure for the image; selecting, by the data processing apparatus, a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image to determine a score for the first image; selecting, by the data processing apparatus, a plurality of candidate images from the other group of images; applying the scoring model to each of the candidate images to determine a respective score for each candidate image, and then selecting a second image from the candidate images, the selection based on a probability that is proportional to a cardinality of the plurality of candidate images and a ranking of the second image relative to each other image in the candidate images based on the respective scores of the images; and determining that the scores for the first image and the second image fail to satisfy a criterion, wherein the criterion requires that a result of the score of the image selected from the positive group of images minus the score of the image selected from the negative group of images exceeds a threshold, updating the scoring model, and storing the updated scoring model.
17. A method performed by a data processing apparatus, comprising: storing data identifying a plurality of training images for a query, wherein each of the training images is classified as being in a positive group of images for the query or a negative group of images for the query according to a respective query specific preference measure for the image; selecting, by the data processing apparatus, a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image to determine a score for the first image; selecting, by the data processing apparatus, a plurality of candidate images from the other group of images; applying the scoring model to each of the candidate images to determine a respective score for each candidate image, and then selecting a second image from the candidate images, the selection based on a probability that is proportional to a cardinality of the plurality of candidate images and a ranking of the second image relative to each other image in the candidate images based on the respective scores of the images; and determining that the scores for the first image and the second image fail to satisfy a criterion, wherein the criterion requires that a result of the score of the image selected from the positive group of images minus the score of the image selected from the negative group of images exceeds a threshold, updating the scoring model, and storing the updated scoring model. 18. The method of claim 17 , wherein selecting the first image comprises: selecting a plurality of first candidate images from either the positive group of images or the negative group of images; applying the scoring model to each of the first candidate images to determine a respective score for each first candidate image; and selecting the first image based on a probability that is proportional to a cardinality of the plurality of first candidate images and a ranking of the first image relative to each other image in the first candidate images based on the respective scores of the first candidate images.