From Eye-blinks to State Construction: Diagnostic Benchmarks for Online
  Representation Learning
  Experiments in classical conditioning show that animals such as rabbits,
pigeons, and dogs can make long temporal associations that enable multi-step
prediction. To replicate this remarkable ability, an agent must construct an
internal state representation that summarizes its interaction history.
Recurrent neural networks can automatically construct state and learn temporal
associations. But the current training methods are prohibitively expensive for
online prediction -- continual learning on every time step -- which is the
focus of this paper. To facilitate research in online prediction, we present
three new diagnostic prediction problems inspired by classical-conditioning
experiments. The proposed problems test the learning capabilities that animals
readily exhibit and highlight the current recurrent learning methods'
limitations. While the proposed problems are nontrivial, they are still
amenable to extensive testing and analysis in the small-compute regime, thereby
enabling researchers to study issues in isolation carefully, ultimately
accelerating progress towards scalable online representation learning methods.
