class_docstr: Retrospective Adversarial Replay for Continual Learning
  https://openreview.net/forum?id=XEoih0EwCwL
  Continual learning is an emerging research challenge in machine learning
  that addresses the problem where models quickly fit the most recently
  trained-on data and are prone to catastrophic forgetting due to
  distribution shifts --- it does this by maintaining a small historical
  replay buffer in replay-based methods.
  To avoid these problems, this paper proposes a method,
  ``Retrospective Adversarial Replay (RAR)'', that synthesizes adversarial
  samples near the forgetting boundary. RAR perturbs a buffered sample
  towards its nearest neighbor drawn from the current task in a latent
  representation space. By replaying such samples, we are able to refine the
  boundary between previous and current tasks, hence combating forgetting and
  reducing bias towards the current task. To mitigate the severity of a small
  replay buffer, we develop a novel MixUp-based strategy to increase replay
  variation by replaying mixed augmentations.
  Combined with RAR, this achieves a holistic framework that helps to
  alleviate catastrophic forgetting. We show that this excels on
  broadly-used benchmarks and outperforms other continual learning baselines
  especially when only a small buffer is used. We conduct a thorough
  ablation study over each key component as well as a hyperparameter
  sensitivity analysis to demonstrate the effectiveness and robustness of RAR.
class_docstr_tok: ["Retrospective","Adversarial","Replay","for","Continual","Learning","https",":","\/\/openreview.net\/forum","?","id=XEoih0EwCwL","Continual","learning","is","an","emerging","research","challenge","in","machine","learning","that","addresses","the","problem","where","models","quickly","fit","the","most","recently","trained-on","data","and","are","prone","to","catastrophic","forgetting","due","to","distribution","shifts","--","-","it","does","this","by","maintaining","a","small","historical","replay","buffer","in","replay-based","methods",".","To","avoid","these","problems",",","this","paper","proposes","a","method",",","``","Retrospective","Adversarial","Replay","(","RAR",")","''",",","that","synthesizes","adversarial","samples","near","the","forgetting","boundary",".","RAR","perturbs","a","buffered","sample","towards","its","nearest","neighbor","drawn","from","the","current","task","in","a","latent","representation","space",".","By","replaying","such","samples",",","we","are","able","to","refine","the","boundary","between","previous","and","current","tasks",",","hence","combating","forgetting","and","reducing","bias","towards","the","current","task",".","To","mitigate","the","severity","of","a","small","replay","buffer",",","we","develop","a","novel","MixUp-based","strategy","to","increase","replay","variation","by","replaying","mixed","augmentations",".","Combined","with","RAR",",","this","achieves","a","holistic","framework","that","helps","to","alleviate","catastrophic","forgetting",".","We","show","that","this","excels","on","broadly-used","benchmarks","and","outperforms","other","continual","learning","baselines","especially","when","only","a","small","buffer","is","used",".","We","conduct","a","thorough","ablation","study","over","each","key","component","as","well","as","a","hyperparameter","sensitivity","analysis","to","demonstrate","the","effectiveness","and","robustness","of","RAR","."]
docstr: :param batch_size_mem: Size of the batch sampled from
                           the bigger buffer
  :param mem_size: Fixed memory size
  :param opt_lr: Learning rate of the internal optimizer
  :param name_ext_layer: Name of the layer to extract features
  :param use_adversarial_replay: Boolean to use or not RAR
  :param beta_coef: float: coef between RAR and Buffer
  :param decay_factor_fgsm: Decay factor of FGSM
  :param epsilon_fgsm: Epsilon for FGSM
  :param iter_fgsm: Number of iterations of FGSM
  :param storage_policy: Storage Policy used for the buffer
docstr_tok: [":","param","batch_size_mem",":","Size","of","the","batch","sampled","from","the","bigger","buffer",":","param","mem_size",":","Fixed","memory","size",":","param","opt_lr",":","Learning","rate","of","the","internal","optimizer",":","param","name_ext_layer",":","Name","of","the","layer","to","extract","features",":","param","use_adversarial_replay",":","Boolean","to","use","or","not","RAR",":","param","beta_coef",":","float",":","coef","between","RAR","and","Buffer",":","param","decay_factor_fgsm",":","Decay","factor","of","FGSM",":","param","epsilon_fgsm",":","Epsilon","for","FGSM",":","param","iter_fgsm",":","Number","of","iterations","of","FGSM",":","param","storage_policy",":","Storage","Policy","used","for","the","buffer"]
code: def __init__(
      self,
      batch_size_mem: int,
      mem_size: int = 200,
      opt_lr: float = 0.1,
      name_ext_layer: str = None,
      use_adversarial_replay: bool = True,
      beta_coef: float = 0.4,
      decay_factor_fgsm: float = 1.0,
      epsilon_fgsm: float = 0.0314,
      iter_fgsm: int = 2,
      storage_policy: Optional["ReservoirSamplingBuffer"] = None,
  ):
      """
      :param batch_size_mem: Size of the batch sampled from
                               the bigger buffer
      :param mem_size: Fixed memory size
      :param opt_lr: Learning rate of the internal optimizer
      :param name_ext_layer: Name of the layer to extract features
      :param use_adversarial_replay: Boolean to use or not RAR
      :param beta_coef: float: coef between RAR and Buffer
      :param decay_factor_fgsm: Decay factor of FGSM
      :param epsilon_fgsm: Epsilon for FGSM
      :param iter_fgsm: Number of iterations of FGSM
      :param storage_policy: Storage Policy used for the buffer
      """
      super().__init__()
      self.mem_size = mem_size
      self.batch_size_mem = batch_size_mem
      self.opt_lr = opt_lr
      self.name_ext_layer = name_ext_layer
  
      self.use_adversarial_replay = use_adversarial_replay
  
      self.beta_coef = beta_coef
      # For Split-CIFAR10: 0.5, 0.1, 0.075 - mem size 200, 500 , 1000
      # Split-CIFAR100 and Split-miniImageNet: 0.4
  
      # FGSM
      self.decay_factor_fgsm = decay_factor_fgsm
      self.epsilon_fgsm = epsilon_fgsm
      self.iter_fgsm = iter_fgsm
  
      self.replay_loader = None
  
      if not self.use_adversarial_replay:
          self.beta_coef = 0
  
      if storage_policy is not None:  # Use other storage policy
          self.storage_policy = storage_policy
          assert storage_policy.max_size == self.mem_size
      else:  # Default
          self.storage_policy = ReservoirSamplingBuffer(
              max_size=self.mem_size
          )
  
file_path: avalanche/avalanche/training/plugins/rar.py
imports: import copy
  from typing import TYPE_CHECKING, Optional
  import numpy
  import torch
  from torch.optim import SGD
  from torch.utils.data.dataloader import DataLoader
  from torchvision.models.feature_extraction import get_graph_node_names, create_feature_extractor
  from avalanche.training.plugins.strategy_plugin import SupervisedPlugin
  from avalanche.training.storage_policy import ReservoirSamplingBuffer
  from avalanche.benchmarks.utils.data_loader import ReplayDataLoader
label_desc: Use image node_id 1 to create a new RARPlugin object from inherited base classes: SupervisedPlugin with example: obj = RARPlugin(batch_size_mem, mem_size, opt_lr, name_ext_layer, use_adversarial_replay, beta_coef, decay_factor_fgsm, epsilon_fgsm, iter_fgsm, storage_policy)