Title: Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning,
Abstract: Interactive reinforcement learning (IRL) extends traditional reinforcement
learning (RL) by allowing an agent to interact with parent-like trainers during
a task. In this paper, we present an IRL approach using dynamic audio-visual
input in terms of vocal commands and hand gestures as feedback. Our
architecture integrates multi-modal information to provide robust commands from
multiple sensory cues along with a confidence value indicating the
trustworthiness of the feedback. The integration process also considers the
case in which the two modalities convey incongruent information. Additionally,
we modulate the influence of sensory-driven feedback in the IRL task using
goal-oriented knowledge in terms of contextual affordances. We implement a
neural network architecture to predict the effect of performed actions with
different objects to avoid failed-states, i.e., states from which it is not
possible to accomplish the task. In our experimental setup, we explore the
interplay of multimodal feedback and task-specific affordances in a robot
cleaning scenario. We compare the learning performance of the agent under four
different conditions: traditional RL, multi-modal IRL, and each of these two
setups with the use of contextual affordances. Our experiments show that the
best performance is obtained by using audio-visual feedback with
affordancemodulated IRL. The obtained results demonstrate the importance of
multi-modal sensory processing integrated with goal-oriented knowledge in IRL
tasks.