class_docstr: Implementation of the Contrastive Cross Entropy Loss.
  
  This implementation follows the SimCLR[0] paper. If you enable the memory
  bank by setting the `memory_bank_size` value > 0 the loss behaves like
  the one described in the MoCo[1] paper.
  
  - [0] SimCLR, 2020, https://arxiv.org/abs/2002.05709
  - [1] MoCo, 2020, https://arxiv.org/abs/1911.05722
  
  Attributes:
      temperature:
          Scale logits by the inverse of the temperature.
      memory_bank_size:
          Size of the memory bank as (num_features, dim) tuple. num_features are the
          number of negative samples stored in the memory bank. If num_features is 0,
          the memory bank is disabled. Use 0 for SimCLR. For MoCo we typically use
          numbers like 4096 or 65536.
          Deprecated: If only a single integer is passed, it is interpreted as the
          number of features and the feature dimension is inferred from the first
          batch stored in the memory bank. Leaving out the feature dimension might
          lead to errors in distributed training.
      gather_distributed:
          If True then negatives from all gpus are gathered before the
          loss calculation. If a memory bank is used and gather_distributed is True,
          then tensors from all gpus are gathered before the memory bank is updated.
  
  Raises:
      ValueError: If abs(temperature) < 1e-8 to prevent divide by zero.
  
  Examples:
  
      >>> # initialize loss function without memory bank
      >>> loss_fn = NTXentLoss(memory_bank_size=0)
      >>>
      >>> # generate two random transforms of images
      >>> t0 = transforms(images)
      >>> t1 = transforms(images)
      >>>
      >>> # feed through SimCLR or MoCo model
      >>> batch = torch.cat((t0, t1), dim=0)
      >>> output = model(batch)
      >>>
      >>> # calculate loss
      >>> loss = loss_fn(output)
code: def __init__(
      self,
      temperature: float = 0.5,
      memory_bank_size: Union[int, Sequence[int]] = 0,
      gather_distributed: bool = False,
  ):
      super().__init__(
          size=memory_bank_size, gather_distributed=gather_distributed
      )
      self.temperature = temperature
      self.gather_distributed = gather_distributed
      self.cross_entropy = nn.CrossEntropyLoss(reduction="mean")
      self.eps = 1e-8
  
      if abs(self.temperature) < self.eps:
          raise ValueError(
              "Illegal temperature: abs({}) < 1e-8".format(
                  self.temperature
              )
          )
      if gather_distributed and not torch_dist.is_available():
          raise ValueError(
              "gather_distributed is True but torch.distributed is not available. "
              "Please set gather_distributed=False or install a torch version with "
              "distributed support."
          )
  
file_path: lightly/lightly/loss/ntx_ent_loss.py
imports: from typing import Sequence, Union
  import torch
  from torch import distributed
  from torch import nn
  from lightly.models.modules.memory_bank import MemoryBankModule
  from lightly.utils import dist
label_desc: Use image node_id 1 to create a new NTXentLoss object from inherited base classes: MemoryBankModule with example: obj = NTXentLoss(temperature, memory_bank_size, gather_distributed)