1. A method for automatically identifying a query object within a visual medium, wherein the method includes one or more processing devices performing operations comprising: receiving, as input to a neural network, a query object and a visual medium including the query object; generating, from a first subset of layers in the neural network, representations of the query object and the visual medium defining features of the query object and the visual medium; generating a plurality of heat maps by applying a second subset of layers in the neural network to the representations, wherein the plurality of heat maps include (i) a first heat map generated from a first combination of a first query object representation and a first visual medium representation and (ii) a second heat map generated from a second combination of a second query object representation and a second visual medium representation; pooling the first heat map and the second heat map to create a set of concatenated layers; generating an output heat map from the set of concatenated layers, wherein the output heat map identifies a location of pixels corresponding to the query object within the visual medium; and generating an updated visual medium using the output heat map to highlight the query object within the visual medium.