Eyes have been shown to play a key role during human social interactions. However, to date, no comprehensive cross-discipline model has provided a framework that can account for uniquely human responses to eye cues. In this review, I present a framework that brings together work on the phylogenetic, ontogenetic, and neural bases of perceiving and responding to eyes. Specifically, I argue for a two-process model: a first process that ensures privileged attention to information encoded in the eyes and is important for the detection of other minds and a second process that permits the decoding of information contained in the eyes concerning another person's emotional and mental states. To some degree, these processes are unique to humans, emerge during different times in infant development, can be mapped onto distinct but interconnected brain regions, and likely serve critical functions in facilitating cooperative interactions in humans. I also present evidence to show that oxytocin is a key modulator of sensitive responding to eye cues. Viewing eyes as windows into other minds can therefore be considered a hallmark feature of human social functioning deeply rooted in our biology.