Transfer of Deep Reactive Policies for MDP Planning
  Domain-independent probabilistic planners input an MDP description in a
factored representation language such as PPDDL or RDDL, and exploit the
specifics of the representation for faster planning. Traditional algorithms
operate on each problem instance independently, and good methods for
transferring experience from policies of other instances of a domain to a new
instance do not exist. Recently, researchers have begun exploring the use of
deep reactive policies, trained via deep reinforcement learning (RL), for MDP
planning domains. One advantage of deep reactive policies is that they are more
amenable to transfer learning.
  In this paper, we present the first domain-independent transfer algorithm for
MDP planning domains expressed in an RDDL representation. Our architecture
exploits the symbolic state configuration and transition function of the domain
(available via RDDL) to learn a shared embedding space for states and
state-action pairs for all problem instances of a domain. We then learn an RL
agent in the embedding space, making a near zero-shot transfer possible, i.e.,
without much training on the new instance, and without using the domain
simulator at all. Experiments on three different benchmark domains underscore
the value of our transfer algorithm. Compared against planning from scratch,
and a state-of-the-art RL transfer algorithm, our transfer solution has
significantly superior learning curves.
