1. A computer-implemented method comprising: receiving a search query that includes a query image of an object; identifying a training image of the object; identifying a particular sub-region of the training image that a visual object recognition engine indicates as matching the query image; determining, by one or more computers, that the particular sub-region of the training image of the object is located within an annotated sub-region of the training image, wherein the annotated sub-region is associated with an annotation; and in response to determining that the particular sub-region of the training image of the object is located within the annotated sub-region of the training image, providing the annotation for output in response to the search query.