Title: Adversarial Imitation via Variational Inverse Reinforcement Learning,
Abstract: We consider a problem of learning the reward and policy from expert examples
under unknown dynamics in high-dimensional scenarios. Our proposed method
builds on the framework of generative adversarial networks and introduces the
empowerment-regularized maximum-entropy inverse reinforcement learning to learn
near-optimal rewards and policies. Empowerment-based regularization prevents
the policy from overfitting expert demonstration, thus leads to a generalized
behavior which results in learning near-optimal rewards. Our method
simultaneously learns empowerment through variational information maximization
along with the reward and policy under the adversarial learning formulation. We
evaluate our approach on various high-dimensional complex control tasks. We
also test our learned rewards in challenging transfer learning problems where
training and testing environments are made to be different from each other in
terms of dynamics or structure. The results show that our proposed method not
only learns near-optimal rewards and policies that are matching expert behavior
but also performs significantly better than state-of-the-art inverse
reinforcement learning algorithms.