Title: Language Bootstrapping: Learning Word Meanings From Perception-Action Association,
Abstract: We address the problem of bootstrapping language acquisition for an
artificial system similarly to what is observed in experiments with human
infants. Our method works by associating meanings to words in manipulation
tasks, as a robot interacts with objects and listens to verbal descriptions of
the interactions. The model is based on an affordance network, i.e., a mapping
between robot actions, robot perceptions, and the perceived effects of these
actions upon objects. We extend the affordance model to incorporate spoken
words, which allows us to ground the verbal symbols to the execution of actions
and the perception of the environment. The model takes verbal descriptions of a
task as the input and uses temporal co-occurrence to create links between
speech utterances and the involved objects, actions, and effects. We show that
the robot is able form useful word-to-meaning associations, even without
considering grammatical structure in the learning process and in the presence
of recognition errors. These word-to-meaning associations are embedded in the
robot's own understanding of its actions. Thus, they can be directly used to
instruct the robot to perform tasks and also allow to incorporate context in
the speech recognition task. We believe that the encouraging results with our
approach may afford robots with a capacity to acquire language descriptors in
their operation's environment as well as to shed some light as to how this
challenging process develops with human infants.