Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.
@cite_5: Inherent in visual scene analysis is a bottleneck associated with the need to sequentially sample locations with foveating eye movements. The concept of a ‘saliency map’ topographically encoding st...
Subsequent research effort has been directed at finding neural correlates of the saliency map and priority map. Some proposed areas for salience computation include the superficial layers of the superior colliculus (sSC) and inferior sections of the pulvinar (PI), and for priority map computation include the frontal eye field (FEF) and deeper layers of the superior colliculus (dSC) @cite_5 . Here, we need to only assume existence of the maps as conceptual mechanisms involved in influencing visual attention and refer the reader to @cite_5 for a recent review.