class_docstr: Implementation of the loss described in 'Emerging Properties in
  Self-Supervised Vision Transformers'. [0]
  
  This implementation follows the code published by the authors. [1]
  It supports global and local image crops. A linear warmup schedule for the
  teacher temperature is implemented to stabilize training at the beginning.
  Centering is applied to the teacher output to avoid model collapse.
  
  - [0]: DINO, 2021, https://arxiv.org/abs/2104.14294
  - [1]: https://github.com/facebookresearch/dino
  
  Attributes:
      output_dim:
          Dimension of the model output.
      warmup_teacher_temp:
          Initial value of the teacher temperature. Should be decreased if the
          training loss does not decrease.
      teacher_temp:
          Final value of the teacher temperature after linear warmup. Values
          above 0.07 result in unstable behavior in most cases. Can be
          slightly increased to improve performance during finetuning.
      warmup_teacher_temp_epochs:
          Number of epochs for the teacher temperature warmup.
      student_temp:
          Temperature of the student.
      center_momentum:
          Momentum term for the center calculation.
  
  Examples:
  
      >>> # initialize loss function
      >>> loss_fn = DINOLoss(128)
      >>>
      >>> # generate a view of the images with a random transform
      >>> view = transform(images)
      >>>
      >>> # embed the view with a student and teacher model
      >>> teacher_out = teacher(view)
      >>> student_out = student(view)
      >>>
      >>> # calculate loss
      >>> loss = loss_fn([teacher_out], [student_out], epoch=0)
code: def __init__(
      self,
      output_dim: int = 65536,
      warmup_teacher_temp: float = 0.04,
      teacher_temp: float = 0.04,
      warmup_teacher_temp_epochs: int = 30,
      student_temp: float = 0.1,
      center_momentum: float = 0.9,
  ):
      super().__init__()
      self.warmup_teacher_temp_epochs = warmup_teacher_temp_epochs
      self.teacher_temp = teacher_temp
      self.student_temp = student_temp
      self.center_momentum = center_momentum
  
      self.register_buffer("center", torch.zeros(1, 1, output_dim))
      # we apply a warm up for the teacher temperature because
      # a too high temperature makes the training instable at the beginning
      self.teacher_temp_schedule = torch.linspace(
          start=warmup_teacher_temp,
          end=teacher_temp,
          steps=warmup_teacher_temp_epochs,
      )
  
file_path: lightly/lightly/loss/dino_loss.py
imports: from typing import List
  import torch
  import torch.distributed
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 1 to create a new DINOLoss object from inherited base classes: nn with example: obj = DINOLoss(output_dim, warmup_teacher_temp, teacher_temp, warmup_teacher_temp_epochs, student_temp, center_momentum)