class_docstr: Learning Without Forgetting.
  
  The method applies knowledge distilllation to mitigate forgetting.
  The teacher is the model checkpoint after the last experience.
docstr: Compute distillation loss between output of the current model and
  and output of the previous (saved) model.
docstr_tok: ["Compute","distillation","loss","between","output","of","the","current","model","and","and","output","of","the","previous","(","saved",")","model","."]
code: def _distillation_loss(self, out, prev_out, active_units):
      """Compute distillation loss between output of the current model and
      and output of the previous (saved) model.
      """
      # we compute the loss only on the previously active units.
      au = list(active_units)
  
      # some people use the crossentropy instead of the KL
      # They are equivalent. We compute
      # kl_div(log_p_curr, p_prev) = p_prev * (log (p_prev / p_curr)) =
      #   p_prev * log(p_prev) - p_prev * log(p_curr).
      # Now, the first term is constant (we don't optimize the teacher),
      # so optimizing the crossentropy and kl-div are equivalent.
      log_p = torch.log_softmax(out[:, au] / self.temperature, dim=1)
      q = torch.softmax(prev_out[:, au] / self.temperature, dim=1)
      res = torch.nn.functional.kl_div(log_p, q, reduction="batchmean")
      return res
  
imports: import copy
  from collections import defaultdict
  from typing import List
  import torch
  import torch.nn.functional
  from avalanche.models import MultiTaskModule, avalanche_forward
label_desc: Use image node_id 2 for calling the LearningWithoutForgetting obj's underlying member method code with example usage: obj._distillation_loss(out, prev_out, active_units) and returns: res