1. A method performed by one or more computers, the method comprising: receiving respective label scores determined by a neural network for each of at least two labels for at least one training example, wherein at least one of the at least two labels for each training example is a positive label for the training example and at least one other of the at least two labels for each training example is a negative label for the training example; determining an error of the neural network based on a semantic ranking loss of the label scores, wherein the semantic ranking loss is determined according to: J = ∑ i = 1 n ⁢ ∑ j = 1 c + ⁢ ∑ k = 1 c - ⁢ D ⁡ ( y c ⁢ + j , y c ⁢ - k ) ⁢ max ⁡ ( 0 , ρ - x i ⁢ W c ⁢ + j + x i ⁢ W c ⁢ - k ) where W is a ranking function of the neural network, n is the number of training examples, x i is an ith training example, c+ is the number of positive labels for the training example x i , c− is the number of negative labels for the training example x i , ρ is a margin for hinge loss, y c+ j is the jth positive label, y c− k is kth negative label, D(y c+ j , y c− k ) is a function that evaluates the semantic distance between two labels, y c+ j and y c− k , x i W c+ j is the label score given to the jth positive label when the ranking function W is used to evaluate the training example x i , and x i W c− k is the label score given to the kth negative label when the ranking function W is used to evaluate the training example x i ; determining updated weights for the neural network based on a gradient of the determined error of the neural network; and applying the updated weights to the neural network to train the neural network.
1. A method performed by one or more computers, the method comprising: receiving respective label scores determined by a neural network for each of at least two labels for at least one training example, wherein at least one of the at least two labels for each training example is a positive label for the training example and at least one other of the at least two labels for each training example is a negative label for the training example; determining an error of the neural network based on a semantic ranking loss of the label scores, wherein the semantic ranking loss is determined according to: J = ∑ i = 1 n ⁢ ∑ j = 1 c + ⁢ ∑ k = 1 c - ⁢ D ⁡ ( y c ⁢ + j , y c ⁢ - k ) ⁢ max ⁡ ( 0 , ρ - x i ⁢ W c ⁢ + j + x i ⁢ W c ⁢ - k ) where W is a ranking function of the neural network, n is the number of training examples, x i is an ith training example, c+ is the number of positive labels for the training example x i , c− is the number of negative labels for the training example x i , ρ is a margin for hinge loss, y c+ j is the jth positive label, y c− k is kth negative label, D(y c+ j , y c− k ) is a function that evaluates the semantic distance between two labels, y c+ j and y c− k , x i W c+ j is the label score given to the jth positive label when the ranking function W is used to evaluate the training example x i , and x i W c− k is the label score given to the kth negative label when the ranking function W is used to evaluate the training example x i ; determining updated weights for the neural network based on a gradient of the determined error of the neural network; and applying the updated weights to the neural network to train the neural network. 8. The method of claim 1 , further comprising: receiving an image; receiving a label corpus comprising labels; scoring, by the neural network, each label from the label corpus with a score representing how well the label describes the image; and annotating the image with a predetermined number of the labels with highest scores from the label corpus.