Using Program Induction to Interpret Transition System Dynamics
  Explaining and reasoning about processes which underlie observed black-box
phenomena enables the discovery of causal mechanisms, derivation of suitable
abstract representations and the formulation of more robust predictions. We
propose to learn high level functional programs in order to represent abstract
models which capture the invariant structure in the observed data. We introduce
the $\pi$-machine (program-induction machine) -- an architecture able to induce
interpretable LISP-like programs from observed data traces. We propose an
optimisation procedure for program learning based on backpropagation, gradient
descent and A* search. We apply the proposed method to two problems: system
identification of dynamical systems and explaining the behaviour of a DQN
agent. Our results show that the $\pi$-machine can efficiently induce
interpretable programs from individual data traces.
