1. A method performed by data processing apparatus, the method comprising: identifying a set of responsive images for a search phrase that includes two or more terms; determining, by one or more processors, interaction rankings for images in the set of responsive images; the interaction ranking of each image being based on a number of user interactions with the image relative to a number of user interactions with other images in the set; creating, by one or more processors, two or more sub-queries based on the search phrase, the sub-queries each being a proper subset of the two or more terms; for each sub-query from the two or more sub-queries: determining, by one or more processors, sub-query model rankings for images in the set of responsive images, the sub-query model ranking of each image being determined based on a sub-query model for the sub-query and features of the images, the sub-query model being an image relevance model for the sub-query; and determining, by one or more processors, a search phrase score for the image relevance model, the search phrase score being a measure of similarity between the interaction rankings of the images and the sub-query model rankings of the images; and selecting, based on the search phrase scores for the sub-queries, one of the sub-query models as a model for the search phrase, the selected sub-query model having a search phrase score that meets a threshold search phrase score.