A Deep Learning Framework for Lifelong Machine Learning
  Humans can learn a variety of concepts and skills incrementally over the
course of their lives while exhibiting many desirable properties, such as
continual learning without forgetting, forward transfer and backward transfer
of knowledge, and learning a new concept or task with only a few examples.
Several lines of machine learning research, such as lifelong machine learning,
few-shot learning, and transfer learning attempt to capture these properties.
However, most previous approaches can only demonstrate subsets of these
properties, often by different complex mechanisms. In this work, we propose a
simple yet powerful unified deep learning framework that supports almost all of
these properties and approaches through one central mechanism. Experiments on
toy examples support our claims. We also draw connections between many
peculiarities of human learning (such as memory loss and "rain man") and our
framework.
  As academics, we often lack resources required to build and train, deep
neural networks with billions of parameters on hundreds of TPUs. Thus, while
our framework is still conceptual, and our experiment results are surely not
SOTA, we hope that this unified lifelong learning framework inspires new work
towards large-scale experiments and understanding human learning in general.
  This paper is summarized in two short YouTube videos:
https://youtu.be/gCuUyGETbTU (part 1) and https://youtu.be/XsaGI01b-1o (part
2).
