Recurrent Environment Simulators
  Models that can simulate how environments change in response to actions can
be used by agents to plan and act efficiently. We improve on previous
environment simulators from high-dimensional pixel observations by introducing
recurrent neural networks that are able to make temporally and spatially
coherent predictions for hundreds of time-steps into the future. We present an
in-depth analysis of the factors affecting performance, providing the most
extensive attempt to advance the understanding of the properties of these
models. We address the issue of computationally inefficiency with a model that
does not need to generate a high-dimensional image at each time-step. We show
that our approach can be used to improve exploration and is adaptable to many
diverse environments, namely 10 Atari games, a 3D car racing environment, and
complex 3D mazes.
