Asking the Right Questions: Learning Interpretable Action Models Through
  Query Answering
  This paper develops a new approach for estimating an interpretable,
relational model of a black-box autonomous agent that can plan and act. Our
main contributions are a new paradigm for estimating such models using a
minimal query interface with the agent, and a hierarchical querying algorithm
that generates an interrogation policy for estimating the agent's internal
model in a vocabulary provided by the user. Empirical evaluation of our
approach shows that despite the intractable search space of possible agent
models, our approach allows correct and scalable estimation of interpretable
agent models for a wide class of black-box autonomous agents. Our results also
show that this approach can use predicate classifiers to learn interpretable
models of planning agents that represent states as images.
