Title: Lifelong Generative Modeling,
Abstract: Lifelong learning is the problem of learning multiple consecutive tasks in a
sequential manner where knowledge gained from previous tasks is retained and
used for future learning. It is essential towards the development of
intelligent machines that can adapt to their surroundings. In this work we
focus on a lifelong learning approach to generative modeling where we
continuously incorporate newly observed distributions into our learnt model. We
do so through a student-teacher Variational Autoencoder architecture which
allows us to learn and preserve all the distributions seen so far without the
need to retain the past data nor the past models. Through the introduction of a
novel cross-model regularizer, inspired by a Bayesian update rule, the student
model leverages the information learnt by the teacher, which acts as a summary
of everything seen till now. The regularizer has the additional benefit of
reducing the effect of catastrophic interference that appears when we learn
over sequences of distributions. We demonstrate its efficacy in learning
sequentially observed distributions as well as its ability to learn a common
latent representation across a complex transfer learning scenario.