Title: Continual Lifelong Learning with Neural Networks: A Review,
Abstract: Humans and animals have the ability to continually acquire, fine-tune, and
transfer knowledge and skills throughout their lifespan. This ability, referred
to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms
that together contribute to the development and specialization of our
sensorimotor skills as well as to the long-term memory consolidation and
retrieval without catastrophic forgetting. Consequently, lifelong learning
capabilities are crucial for autonomous agents interacting in the real world
and processing continuous streams of information. However, lifelong learning
remains a long-standing challenge for machine learning and neural network
models since the continual acquisition of incrementally available information
from non-stationary data distributions generally leads to catastrophic
forgetting or interference. This limitation represents a major drawback for
state-of-the-art deep neural network models that typically learn
representations from stationary batches of training data, thus without
accounting for situations in which information becomes incrementally available
over time. In this review, we critically summarize the main challenges linked
to lifelong learning for artificial learning systems and compare existing
neural network approaches that alleviate, to different extents, catastrophic
forgetting. We discuss well-established and emerging research motivated by
lifelong learning factors in biological systems such as structural plasticity,
memory replay, curriculum and transfer learning, intrinsic motivation, and
multisensory integration.