class_docstr: Memory Aware Synapses (MAS) plugin.
  
  Similarly to EWC, the MAS plugin computes the importance of each
  parameter at the end of each experience. The approach computes
  importance via a second pass on the dataset. MAS does not require
  supervision and estimates importance using the gradients of the
  L2 norm of the output. Importance is then used to add a penalty
  term to the loss function.
  
  Technique introduced in:
  "Memory Aware Synapses: Learning what (not) to forget"
  by Aljundi et. al (2018).
  
  Implementation based on FACIL, as in:
  https://github.com/mmasana/FACIL/blob/master/src/approach/mas.py
class_docstr_tok: ["Memory","Aware","Synapses","(","MAS",")","plugin",".","Similarly","to","EWC",",","the","MAS","plugin","computes","the","importance","of","each","parameter","at","the","end","of","each","experience",".","The","approach","computes","importance","via","a","second","pass","on","the","dataset",".","MAS","does","not","require","supervision","and","estimates","importance","using","the","gradients","of","the","L2","norm","of","the","output",".","Importance","is","then","used","to","add","a","penalty","term","to","the","loss","function",".","Technique","introduced","in",":","''","Memory","Aware","Synapses",":","Learning","what","(","not",")","to","forget","''","by","Aljundi","et",".","al","(","2018",")",".","Implementation","based","on","FACIL",",","as","in",":","https",":","\/\/github.com\/mmasana\/FACIL\/blob\/master\/src\/approach\/mas.py"]
code: def _get_importance(self, strategy):
      # Initialize importance matrix
      importance = dict(zerolike_params_dict(strategy.model))
  
      if not strategy.experience:
          raise ValueError("Current experience is not available")
  
      if strategy.experience.dataset is None:
          raise ValueError("Current dataset is not available")
  
      # Do forward and backward pass to accumulate L2-loss gradients
      strategy.model.train()
      collate_fn = (
          strategy.experience.dataset.collate_fn
          if hasattr(strategy.experience.dataset, "collate_fn")
          else None
      )
      dataloader = DataLoader(
          strategy.experience.dataset,
          batch_size=strategy.train_mb_size,
          collate_fn=collate_fn,
      )  # type: ignore
  
      # Progress bar
      if self.verbose:
          print("Computing importance")
          dataloader = tqdm(dataloader)
  
      for _, batch in enumerate(dataloader):
          # Get batch
          if len(batch) == 2 or len(batch) == 3:
              x, _, t = batch[0], batch[1], batch[-1]
          else:
              raise ValueError("Batch size is not valid")
  
          # Move batch to device
          x = x.to(strategy.device)
  
          # Forward pass
          strategy.optimizer.zero_grad()
          out = avalanche_forward(strategy.model, x, t)
  
          # Average L2-Norm of the output
          loss = torch.norm(out, p="fro", dim=1).pow(2).mean()
          loss.backward()
  
          # Accumulate importance
          for name, param in strategy.model.named_parameters():
              if param.requires_grad:
                  # In multi-head architectures, the gradient is going
                  # to be None for all the heads different from the
                  # current one.
                  if param.grad is not None:
                      importance[name].data += param.grad.abs()
  
      # Normalize importance
      for k in importance.keys():
          importance[k].data /= float(len(dataloader))
  
      return importance
  
file_path: avalanche/avalanche/training/plugins/mas.py
imports: from tqdm.auto import tqdm
  from typing import Dict, Union
  from torch.utils.data import DataLoader
  import torch
  from avalanche.models.utils import avalanche_forward
  from avalanche.training.plugins.strategy_plugin import SupervisedPlugin
  from avalanche.training.utils import copy_params_dict, zerolike_params_dict, ParamData
label_desc: Use image node_id 2 for calling the MASPlugin obj's underlying member method code with example usage: obj._get_importance(strategy) and returns: importance