1. A computer implemented method comprising: obtaining, by one or more computers and for a first object in an image, a first set of text labels using visual data extracted from the image; obtaining, by the one or more computers and for a second object in the image, a second set of text labels using the visual data extracted from the image; generating, by the one or more computers and for various pairs of text labels, a relation score based on a number of co-occurrences of the text labels in text of web pages, wherein each pair of text labels includes a text label from the first set and a text label from the second set; and labeling, by the one or more computers, the image with a given pair of text labels, from among the various pairs of text labels, based on solving a global optimization problem utilizing at least the relation score, wherein the relation score for the given pair of text labels meets a specified threshold relation score.