class_docstr: Implementation of the Decoupled Contrastive Learning Loss from
  Decoupled Contrastive Learning [0].
  
  This code implements Equation 6 in [0], including the sum over all images `i`
  and views `k`. The loss is reduced to a mean loss over the mini-batch.
  The implementation was inspired by [1].
  
  - [0] Chun-Hsiao Y. et. al., 2021, Decoupled Contrastive Learning https://arxiv.org/abs/2110.06848
  - [1] https://github.com/raminnakhli/Decoupled-Contrastive-Learning
  
  Attributes:
      temperature:
          Similarities are scaled by inverse temperature.
      weight_fn:
          Weighting function `w` from the paper. Scales the loss between the
          positive views (views from the same image). No weighting is performed
          if weight_fn is None. The function must take the two input tensors
          passed to the forward call as input and return a weight tensor. The
          returned weight tensor must have the same length as the input tensors.
      gather_distributed:
          If True then negatives from all gpus are gathered before the
          loss calculation.
  
  Examples:
  
      >>> loss_fn = DCLLoss(temperature=0.07)
      >>>
      >>> # generate two random transforms of images
      >>> t0 = transforms(images)
      >>> t1 = transforms(images)
      >>>
      >>> # embed images using some model, for example SimCLR
      >>> out0 = model(t0)
      >>> out1 = model(t1)
      >>>
      >>> # calculate loss
      >>> loss = loss_fn(out0, out1)
      >>>
      >>> # you can also add a custom weighting function
      >>> weight_fn = lambda out0, out1: torch.sum((out0 - out1) ** 2, dim=1)
      >>> loss_fn = DCLLoss(weight_fn=weight_fn)
docstr: Calculates DCL loss for out0 with respect to its positives in out1
  and the negatives in out1, out0_all, and out1_all.
  
  This code implements Equation 6 in [0], including the sum over all images `i`
  but with `k` fixed at 0.
  
  Args:
      out0:
          Output projections of the first set of transformed images.
          Shape: (batch_size, embedding_size)
      out1:
          Output projections of the second set of transformed images.
          Shape: (batch_size, embedding_size)
      out0_all:
          Output projections of the first set of transformed images from
          all distributed processes/gpus. Should be equal to out0 in an
          undistributed setting.
          Shape (batch_size * world_size, embedding_size)
      out1_all:
          Output projections of the second set of transformed images from
          all distributed processes/gpus. Should be equal to out1 in an
          undistributed setting.
          Shape (batch_size * world_size, embedding_size)
  
  Returns:
      Mean loss over the mini-batch.
docstr_tok: ["Calculates","DCL","loss","for","out0","with","respect","to","its","positives","in","out1","and","the","negatives","in","out1",",","out0_all",",","and","out1_all",".","This","code","implements","Equation","6","in","[","0","]",",","including","the","sum","over","all","images","`","i","`","but","with","`","k","`","fixed","at","0",".","Args",":","out0",":","Output","projections","of","the","first","set","of","transformed","images",".","Shape",":","(","batch_size",",","embedding_size",")","out1",":","Output","projections","of","the","second","set","of","transformed","images",".","Shape",":","(","batch_size",",","embedding_size",")","out0_all",":","Output","projections","of","the","first","set","of","transformed","images","from","all","distributed","processes\/gpus",".","Should","be","equal","to","out0","in","an","undistributed","setting",".","Shape","(","batch_size","*","world_size",",","embedding_size",")","out1_all",":","Output","projections","of","the","second","set","of","transformed","images","from","all","distributed","processes\/gpus",".","Should","be","equal","to","out1","in","an","undistributed","setting",".","Shape","(","batch_size","*","world_size",",","embedding_size",")","Returns",":","Mean","loss","over","the","mini-batch","."]
code: def _loss(self, out0, out1, out0_all, out1_all):
      """Calculates DCL loss for out0 with respect to its positives in out1
      and the negatives in out1, out0_all, and out1_all.
  
      This code implements Equation 6 in [0], including the sum over all images `i`
      but with `k` fixed at 0.
  
      Args:
          out0:
              Output projections of the first set of transformed images.
              Shape: (batch_size, embedding_size)
          out1:
              Output projections of the second set of transformed images.
              Shape: (batch_size, embedding_size)
          out0_all:
              Output projections of the first set of transformed images from
              all distributed processes/gpus. Should be equal to out0 in an
              undistributed setting.
              Shape (batch_size * world_size, embedding_size)
          out1_all:
              Output projections of the second set of transformed images from
              all distributed processes/gpus. Should be equal to out1 in an
              undistributed setting.
              Shape (batch_size * world_size, embedding_size)
  
      Returns:
          Mean loss over the mini-batch.
      """
      # create diagonal mask that only selects similarities between
      # representations of the same images
      batch_size = out0.shape[0]
      if self.gather_distributed and dist.world_size() > 1:
          diag_mask = dist.eye_rank(batch_size, device=out0.device)
      else:
          diag_mask = torch.eye(
              batch_size, device=out0.device, dtype=torch.bool
          )
  
      # calculate similarities
      # here n = batch_size and m = batch_size * world_size.
      sim_00 = (
          torch.einsum("nc,mc->nm", out0, out0_all) / self.temperature
      )
      sim_01 = (
          torch.einsum("nc,mc->nm", out0, out1_all) / self.temperature
      )
  
      positive_loss = -sim_01[diag_mask]
      if self.weight_fn:
          positive_loss = positive_loss * self.weight_fn(out0, out1)
  
      # remove simliarities between same views of the same image
      sim_00 = sim_00[~diag_mask].view(batch_size, -1)
      # remove similarities between different views of the same images
      # this is the key difference compared to NTXentLoss
      sim_01 = sim_01[~diag_mask].view(batch_size, -1)
  
      negative_loss_00 = torch.logsumexp(sim_00, dim=1)
      negative_loss_01 = torch.logsumexp(sim_01, dim=1)
      return (
          positive_loss + negative_loss_00 + negative_loss_01
      ).mean()
  
file_path: lightly/lightly/loss/dcl_loss.py
imports: from functools import partial
  from typing import Callable, Optional
  import torch
  from torch import Tensor
  from torch import distributed
  from torch import nn
  from lightly.utils import dist
label_desc: Use image node_id 3 for calling the DCLLoss obj's underlying member method code with example usage: obj._loss(out0, out1, out0_all, out1_all) and returns: unknown