class_docstr: Implementation of the CO2 regularizer [0] for self-supervised learning.
  
  [0] CO2, 2021, https://arxiv.org/abs/2010.02217
  
  Attributes:
      alpha:
          Weight of the regularization term.
      t_consistency:
          Temperature used during softmax calculations.
      memory_bank_size:
          Size of the memory bank as (num_features, dim) tuple. num_features is the
          number of negatives stored in the bank. If set to 0, the memory bank is
          disabled. Deprecated: If only a single integer is passed, it is interpreted
          as the number of features and the feature dimension is inferred from the
          first batch stored in the memory bank. Leaving out the feature dimension
          might lead to errors in distributed training.
  
  Examples:
      >>> # initialize loss function for MoCo
      >>> loss_fn = NTXentLoss(memory_bank_size=(4096, 128))
      >>>
      >>> # initialize CO2 regularizer
      >>> co2 = CO2Regularizer(alpha=1.0, memory_bank_size=(4096, 128))
      >>>
      >>> # generate two random trasnforms of images
      >>> t0 = transforms(images)
      >>> t1 = transforms(images)
      >>>
      >>> # feed through the MoCo model
      >>> out0, out1 = model(t0, t1)
      >>>
      >>> # calculate loss and apply regularizer
      >>> loss = loss_fn(out0, out1) + co2(out0, out1)
docstr: Computes the CO2 regularization term for two model outputs.
  
  Args:
      out0:
          Output projections of the first set of transformed images.
      out1:
          Output projections of the second set of transformed images.
  
  Returns:
      The regularization term multiplied by the weight factor alpha.
docstr_tok: ["Computes","the","CO2","regularization","term","for","two","model","outputs",".","Args",":","out0",":","Output","projections","of","the","first","set","of","transformed","images",".","out1",":","Output","projections","of","the","second","set","of","transformed","images",".","Returns",":","The","regularization","term","multiplied","by","the","weight","factor","alpha","."]
code: def forward(self, out0: torch.Tensor, out1: torch.Tensor):
      """Computes the CO2 regularization term for two model outputs.
  
      Args:
          out0:
              Output projections of the first set of transformed images.
          out1:
              Output projections of the second set of transformed images.
  
      Returns:
          The regularization term multiplied by the weight factor alpha.
  
      """
  
      # normalize the output to length 1
      out0 = torch.nn.functional.normalize(out0, dim=1)
      out1 = torch.nn.functional.normalize(out1, dim=1)
  
      # ask memory bank for negative samples and extend it with out1 if
      # out1 requires a gradient, otherwise keep the same vectors in the
      # memory bank (this allows for keeping the memory bank constant e.g.
      # for evaluating the loss on the test set)
      # if the memory_bank size is 0, negatives will be None
      out1, negatives = super(CO2Regularizer, self).forward(
          out1, update=True
      )
  
      # get log probabilities
      p = self._get_pseudo_labels(out0, out1, negatives)
      q = self._get_pseudo_labels(out1, out0, negatives)
  
      # calculate symmetrized kullback leibler divergence
      if self.log_target:
          div = self.kl_div(p, q) + self.kl_div(q, p)
      else:
          # can't use log_target because of early torch version
          div = self.kl_div(p, torch.exp(q)) + self.kl_div(
              q, torch.exp(p)
          )
  
      return self.alpha * 0.5 * div
  
file_path: lightly/lightly/loss/regularizer/co2.py
imports: from typing import Sequence, Union
  import torch
  from lightly.models.modules.memory_bank import MemoryBankModule
label_desc: Use image node_id 3 for calling the CO2Regularizer obj's underlying member method code with example usage: obj.forward(out0, out1) and returns: unknown