Title: Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation,
Abstract: Vision-language navigation (VLN) is the task of navigating an embodied agent
to carry out natural language instructions inside real 3D environments. In this
paper, we study how to address three critical challenges for this task: the
cross-modal grounding, the ill-posed feedback, and the generalization problems.
First, we propose a novel Reinforced Cross-Modal Matching (RCM) approach that
enforces cross-modal grounding both locally and globally via reinforcement
learning (RL). Particularly, a matching critic is used to provide an intrinsic
reward to encourage global matching between instructions and trajectories, and
a reasoning navigator is employed to perform cross-modal grounding in the local
visual scene. Evaluation on a VLN benchmark dataset shows that our RCM model
significantly outperforms existing methods by 10% on SPL and achieves the new
state-of-the-art performance. To improve the generalizability of the learned
policy, we further introduce a Self-Supervised Imitation Learning (SIL) method
to explore unseen environments by imitating its own past, good decisions. We
demonstrate that SIL can approximate a better and more efficient policy, which
tremendously minimizes the success rate performance gap between seen and unseen
environments (from 30.7% to 11.7%).