10. A method as claim 9 recites, wherein re-ranking the ranked set of sentences by using the deep multimodal similarity model comprising the image model and the text model to generate caption sentences comprises: generating an image vector from the image using the image model; and generating a plurality of text vectors from the ranked set of sentences using the text model.
10. A method as claim 9 recites, wherein re-ranking the ranked set of sentences by using the deep multimodal similarity model comprising the image model and the text model to generate caption sentences comprises: generating an image vector from the image using the image model; and generating a plurality of text vectors from the ranked set of sentences using the text model. 11. A method as claim 10 recites, wherein generating the plurality of text vectors from the image comprises: converting each word of the plurality of sentences of the ranked set of sentences to a letter-trigram count vector; and propagating forward the letter-trigram count vector through a deep convolutional neural network to produce a semantic vector.