class_docstr: Implementation of the CO2 regularizer [0] for self-supervised learning.
  
  [0] CO2, 2021, https://arxiv.org/abs/2010.02217
  
  Attributes:
      alpha:
          Weight of the regularization term.
      t_consistency:
          Temperature used during softmax calculations.
      memory_bank_size:
          Size of the memory bank as (num_features, dim) tuple. num_features is the
          number of negatives stored in the bank. If set to 0, the memory bank is
          disabled. Deprecated: If only a single integer is passed, it is interpreted
          as the number of features and the feature dimension is inferred from the
          first batch stored in the memory bank. Leaving out the feature dimension
          might lead to errors in distributed training.
  
  Examples:
      >>> # initialize loss function for MoCo
      >>> loss_fn = NTXentLoss(memory_bank_size=(4096, 128))
      >>>
      >>> # initialize CO2 regularizer
      >>> co2 = CO2Regularizer(alpha=1.0, memory_bank_size=(4096, 128))
      >>>
      >>> # generate two random trasnforms of images
      >>> t0 = transforms(images)
      >>> t1 = transforms(images)
      >>>
      >>> # feed through the MoCo model
      >>> out0, out1 = model(t0, t1)
      >>>
      >>> # calculate loss and apply regularizer
      >>> loss = loss_fn(out0, out1) + co2(out0, out1)
code: def __init__(
      self,
      alpha: float = 1,
      t_consistency: float = 0.05,
      memory_bank_size: Union[int, Sequence[int]] = 0,
  ):
      super(CO2Regularizer, self).__init__(size=memory_bank_size)
      # try-catch the KLDivLoss construction for backwards compatability
      self.log_target = True
      try:
          self.kl_div = torch.nn.KLDivLoss(
              reduction="batchmean", log_target=True
          )
      except TypeError:
          self.log_target = False
          self.kl_div = torch.nn.KLDivLoss(reduction="batchmean")
  
      self.t_consistency = t_consistency
      self.alpha = alpha
  
file_path: lightly/lightly/loss/regularizer/co2.py
imports: from typing import Sequence, Union
  import torch
  from lightly.models.modules.memory_bank import MemoryBankModule
label_desc: Use image node_id 1 to create a new CO2Regularizer object from inherited base classes: MemoryBankModule with example: obj = CO2Regularizer(alpha, t_consistency, memory_bank_size)