Addressing the Stability-Plasticity Dilemma via Knowledge-Aware   Continual Learning
Current methods in continual learning (CL) tend to focus on alleviating catastrophic forgetting of previous tasks. This hinders balancing other CL desiderata such as forward transfer, and memory and computation efficiency. CL desiderata become much more competing when a model operates on class-Incremental Learning (class-IL) without accessing past data since it is prone to ambiguities between old and new classes. In this paper, we present Knowledge-Aware coNtinual learner (KAN) that attempts to study the stability-plasticity dilemma to balance CL desiderata in class-IL. In particular, KAN is a new task-specific components method based on dynamic sparse training that introduces reusability and selective transfer of past knowledge in this class of methods. KAN selectively reuses relevant knowledge while addressing class ambiguity, preserves old knowledge, and utilizes the model capacity efficiently. Experiments show the effectiveness of KAN in providing models with multiple CL desirable properties, outperforming state-of-the-art methods on various challenging benchmarks.