Title: Self-Supervised Vision-Based Detection of the Active Speaker as a Prerequisite for Socially-Aware Language Acquisition,
Abstract: This paper presents a self-supervised method for detecting the active speaker
in a multi-person spoken interaction scenario. We argue that this capability is
a fundamental prerequisite for any artificial cognitive system attempting to
acquire language in social settings. Our methods are able to detect an
arbitrary number of possibly overlapping active speakers based exclusively on
visual information about their face. Our methods do not rely on external
annotations, thus complying with cognitive development. Instead, they use
information from the auditory modality to support learning in the visual
domain. The methods have been extensively evaluated on a large multi-person
face-to-face interaction dataset. The results reach an accuracy of 80% on a
multi-speaker setting. We believe this system represents an essential component
of any artificial cognitive system or robotic platform engaging in social
interaction.