class_docstr: Implementation of the VICRegL loss from VICRegL paper [0].
  
  This implementation follows the code published by the authors [1].
  
  - [0]: VICRegL, 2022, https://arxiv.org/abs/2210.01571
  - [1]: https://github.com/facebookresearch/VICRegL
  
  Attributes:
      lambda_param:
          Coefficient for the invariance term of the loss.
      mu_param:
          Coefficient for the variance term of the loss.
      nu_param:
          Coefficient for the covariance term of the loss.
      alpha:
          Coefficient to weight global with local loss. The final loss is computed as
          (self.alpha * global_loss + (1-self.alpha) * local_loss).
      gather_distributed:
          If True then the cross-correlation matrices from all gpus are gathered and
          summed before the loss calculation.
      eps:
          Epsilon for numerical stability.
      num_matches:
          Number of local features to match using nearest neighbors.
  
  Examples:
  
      >>> # initialize loss function
      >>> criterion = VICRegLLoss()
      >>> transform = VICRegLTransform(n_global_views=2, n_local_views=4)
      >>>
      >>> # generate two random transforms of images
      >>> views_and_grids = transform(images)
      >>> views = views_and_grids[:6] # 2 global views + 4 local views
      >>> grids = views_and_grids[6:]
      >>>
      >>> # feed through model images
      >>> features = [model(view) for view in views]
      >>>
      >>> # calculate loss
      >>> loss = criterion(
      ...     global_view_features=features[:2],
      ...     global_view_grids=grids[:2],
      ...     local_view_features=features[2:],
      ...     local_view_grids=grids[2:],
      ... )
class_docstr_tok: ["Implementation","of","the","VICRegL","loss","from","VICRegL","paper","[","0","]",".","This","implementation","follows","the","code","published","by","the","authors","[","1","]",".","-","[","0","]",":","VICRegL",",","2022",",","https",":","\/\/arxiv.org\/abs\/2210.01571","-","[","1","]",":","https",":","\/\/github.com\/facebookresearch\/VICRegL","Attributes",":","lambda_param",":","Coefficient","for","the","invariance","term","of","the","loss",".","mu_param",":","Coefficient","for","the","variance","term","of","the","loss",".","nu_param",":","Coefficient","for","the","covariance","term","of","the","loss",".","alpha",":","Coefficient","to","weight","global","with","local","loss",".","The","final","loss","is","computed","as","(","self.alpha","*","global_loss","+","(","1-self.alpha",")","*","local_loss",")",".","gather_distributed",":","If","True","then","the","cross-correlation","matrices","from","all","gpus","are","gathered","and","summed","before","the","loss","calculation",".","eps",":","Epsilon","for","numerical","stability",".","num_matches",":","Number","of","local","features","to","match","using","nearest","neighbors",".","Examples",":",">",">",">","#","initialize","loss","function",">",">",">","criterion","=","VICRegLLoss","(",")",">",">",">","transform","=","VICRegLTransform","(","n_global_views=2",",","n_local_views=4",")",">",">",">",">",">",">","#","generate","two","random","transforms","of","images",">",">",">","views_and_grids","=","transform","(","images",")",">",">",">","views","=","views_and_grids","[",":6","]","#","2","global","views","+","4","local","views",">",">",">","grids","=","views_and_grids","[","6",":","]",">",">",">",">",">",">","#","feed","through","model","images",">",">",">","features","=","[","model","(","view",")","for","view","in","views","]",">",">",">",">",">",">","#","calculate","loss",">",">",">","loss","=","criterion","(","...","global_view_features=features","[",":2","]",",","...","global_view_grids=grids","[",":2","]",",","...","local_view_features=features","[","2",":","]",",","...","local_view_grids=grids","[","2",":","]",",","...",")"]
code: def __init__(
      self,
      lambda_param: float = 25.0,
      mu_param: float = 25.0,
      nu_param: float = 1.0,
      alpha: float = 0.75,
      gather_distributed: bool = False,
      eps: float = 0.0001,
      num_matches: Tuple[int, int] = (20, 4),
  ):
      super(VICRegLLoss, self).__init__()
      self.alpha = alpha
      self.num_matches = num_matches
      self.lambda_param = lambda_param
      self.mu_param = mu_param
      self.nu_param = nu_param
      self.eps = eps
      self.gather_distributed = gather_distributed
      # Note: We multiply nu_param by 0.5 because the implementations of the VICReg
      # covariance loss differ by a factor of 0.5 between the original VICReg and
      # VICRegL codebases. See:
      # - VICReg: https://github.com/facebookresearch/vicreg/blob/4e12602fd495af83efd1631fbe82523e6db092e0/main_vicreg.py#L211-L213
      # - VICRegL: https://github.com/facebookresearch/VICRegL/blob/803ae4c8cd1649a820f03afb4793763e95317620/main_vicregl.py#L308-L312
      self.vicreg_loss = VICRegLoss(
          lambda_param=lambda_param,
          mu_param=mu_param,
          nu_param=0.5 * nu_param,
          eps=eps,
          gather_distributed=gather_distributed,
      )
  
file_path: lightly/lightly/loss/vicregl_loss.py
imports: from typing import Optional, Sequence, Tuple
  import torch
  import torch.distributed
  from torch import Tensor
  from lightly.loss.vicreg_loss import VICRegLoss, covariance_loss, invariance_loss, variance_loss
  from lightly.models.utils import nearest_neighbors
  from lightly.utils.dist import gather
label_desc: Use image node_id 1 to create a new VICRegLLoss object from inherited base classes: torch.nn with example: obj = VICRegLLoss(lambda_param, mu_param, nu_param, alpha, gather_distributed, eps, num_matches)