Difference between text-based image retrieval and natural language object retrieval
Disclaimer: I can only answer for the NLP part since I'm no expert for image processing.
I assume that text-based image retrieval is the task of finding the image (or the part of an image) which corresponds to a short text which exclusively describes the object. Practically it means that any content word (i.e. excluding grammatical words like determiners) in the text refers directly to the object: &quot;a bike&quot;, &quot;a black cat&quot;, &quot;the red car&quot;, etc. For a ML process it means that there's nothing to analyze in the text, every word can directly be associated with a characteristic of the image.
By contrast Natural Language object retrieval involves analyzing the text. For instance &quot;the cat on the left of the picture&quot; is different than &quot;the picture on the left of the cat&quot;, even though the words are the same. Additionally there can be different ways to refer to the same object: &quot;the book at the left of the shelf&quot; may be the same as &quot;the leftmost book&quot; or &quot;the book next to the green book&quot;. There are usually many ways to express the same meaning with language, and that makes the task much more complex. Additionally I would assume that mapping positional descriptions to the image characteristics can be tricky: &quot;the man behind the tree&quot; or &quot;the second bridge&quot; in a 2D image requires the model to &quot;understand&quot; depth. In a picture with two dogs, &quot;the small dog&quot; requires the model to &quot;understand&quot; size relation between objects. Humans intuitively know how to interpret these sentences, but for a machine Natural Language Understanding hasn't been solved yet (it might never be).