19. The system of claim 17 , wherein determining the updated weights of the neural network comprises training the neural network with gradient descent backpropagation.
17. A system comprising: one or more computers and one or more storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving respective label scores determined by a neural network for at least two labels for at least one training example wherein at least one of the at least two labels for each training example is a positive label for the training example and at least one other of the at least two labels is a negative label for the training example; determining an error of the neural network based on a semantic ranking loss of the label scores, wherein the semantic ranking loss is determined according to: J = ∑ i = 1 n ⁢ ∑ j = 1 c + ⁢ ∑ k = 1 c - ⁢ D ⁡ ( y c ⁢ + j , y c ⁢ - k ) ⁢ max ⁡ ( 0 , ρ - x i ⁢ W c ⁢ + j + x i ⁢ W c ⁢ - k ) where W is a ranking function of the neural network, n is the number of training examples, x i is an ith training example, c+ is the number of positive labels for the training example x i , c− is the number of negative labels for the training example x i , ρ is a margin for hinge loss, y c+ j is the jth positive label, y c− k is kth negative label, D(y c+ j , y c− k ) is a function that evaluates the semantic distance between two labels, y c+ j and y c− k , x i W c+ j is the label score given to the jth positive label when the ranking function W is used to evaluate the training example x i , and x i W c− k is the label score given to the kth negative label when the ranking function W is used to evaluate the training example x i ; determining updated weights for the neural network based on a gradient of the determined error of the neural network; and applying the updated weights to the neural network to train the neural network.