1. A computer-implemented method comprising: receiving a search query from a particular user, wherein the search query includes a query image; obtaining a set of submitted images that (i) were submitted by one or more other users, and (ii) are identified as similar to the query image that is included in the search query; determining, for each submitted image of the set of submitted images that were submitted by the one or more other users and are identified as similar to the query image that is included in the search query, (i) a query relevance score that reflects a relevance of the submitted image to the query image and (ii) an annotation associated with the submitted image; grouping the submitted images of the set of submitted images by the annotations associated with the submitted images that were submitted by the one or more other users; for each group of images in the set of submitted images that were submitted by the one or more other users, aggregating the query relevance scores that reflect the relevance of the submitted images in the group of images to the query image; selecting a particular group of images based on the aggregated query relevance scores that reflect the relevance of the submitted images in the groups of images to the query image; selecting, as a representative annotation, the annotation associated with the particular group of images that was selected based on the aggregated query relevance scores that reflect the relevance of the groups of images to the query image; and providing the representative annotation for output to the particular user on a search results page in response to the search query.