class_docstr: Memory bank implementation
  
  This is a parent class to all loss functions implemented by the lightly
  Python package. This way, any loss can be used with a memory bank if
  desired.
  
  Attributes:
      size:
          Size of the memory bank as (num_features, dim) tuple. If num_features is 0
          then the memory bank is disabled. Deprecated: If only a single integer is
          passed, it is interpreted as the number of features and the feature
          dimension is inferred from the first batch stored in the memory bank.
          Leaving out the feature dimension might lead to errors in distributed
          training.
      gather_distributed:
          If True then negatives from all gpus are gathered before the memory bank
          is updated. This results in more frequent updates of the memory bank and
          keeps the memory bank contents independent of the number of gpus. But it has
          the drawback that synchronization between processes is required and
          diversity of the memory bank content is reduced.
      feature_dim_first:
          If True, the memory bank returns features with shape (dim, num_features).
          If False, the memory bank returns features with shape (num_features, dim).
  
  Examples:
      >>> class MyLossFunction(MemoryBankModule):
      >>>
      >>>     def __init__(self, memory_bank_size: Tuple[int, int] = (2 ** 16, 128)):
      >>>         super().__init__(memory_bank_size)
      >>>
      >>>     def forward(self, output: Tensor, labels: Union[Tensor, None] = None):
      >>>         output, negatives = super().forward(output)
      >>>
      >>>         if negatives is not None:
      >>>             # evaluate loss with negative samples
      >>>         else:
      >>>             # evaluate loss without negative samples
class_docstr_tok: ["Memory","bank","implementation","This","is","a","parent","class","to","all","loss","functions","implemented","by","the","lightly","Python","package",".","This","way",",","any","loss","can","be","used","with","a","memory","bank","if","desired",".","Attributes",":","size",":","Size","of","the","memory","bank","as","(","num_features",",","dim",")","tuple",".","If","num_features","is","0","then","the","memory","bank","is","disabled",".","Deprecated",":","If","only","a","single","integer","is","passed",",","it","is","interpreted","as","the","number","of","features","and","the","feature","dimension","is","inferred","from","the","first","batch","stored","in","the","memory","bank",".","Leaving","out","the","feature","dimension","might","lead","to","errors","in","distributed","training",".","gather_distributed",":","If","True","then","negatives","from","all","gpus","are","gathered","before","the","memory","bank","is","updated",".","This","results","in","more","frequent","updates","of","the","memory","bank","and","keeps","the","memory","bank","contents","independent","of","the","number","of","gpus",".","But","it","has","the","drawback","that","synchronization","between","processes","is","required","and","diversity","of","the","memory","bank","content","is","reduced",".","feature_dim_first",":","If","True",",","the","memory","bank","returns","features","with","shape","(","dim",",","num_features",")",".","If","False",",","the","memory","bank","returns","features","with","shape","(","num_features",",","dim",")",".","Examples",":",">",">",">","class","MyLossFunction","(","MemoryBankModule",")",":",">",">",">",">",">",">","def","__init__","(","self",",","memory_bank_size",":","Tuple","[","int",",","int","]","=","(","2","*","*","16",",","128",")",")",":",">",">",">","super","(",")",".__init__","(","memory_bank_size",")",">",">",">",">",">",">","def","forward","(","self",",","output",":","Tensor",",","labels",":","Union","[","Tensor",",","None","]","=","None",")",":",">",">",">","output",",","negatives","=","super","(",")",".forward","(","output",")",">",">",">",">",">",">","if","negatives","is","not","None",":",">",">",">","#","evaluate","loss","with","negative","samples",">",">",">","else",":",">",">",">","#","evaluate","loss","without","negative","samples"]
docstr: Query memory bank for additional negative samples
  
  Args:
      output:
          The output of the model.
      labels:
          Should always be None, will be ignored.
      update:
          If True, the memory bank will be updated with the current output.
  
  Returns:
      The output if the memory bank is of size 0, otherwise the output
      and the entries from the memory bank. Entries from the memory bank have
      shape (dim, num_features) if feature_dim_first is True and
      (num_features, dim) otherwise.
docstr_tok: ["Query","memory","bank","for","additional","negative","samples","Args",":","output",":","The","output","of","the","model",".","labels",":","Should","always","be","None",",","will","be","ignored",".","update",":","If","True",",","the","memory","bank","will","be","updated","with","the","current","output",".","Returns",":","The","output","if","the","memory","bank","is","of","size","0",",","otherwise","the","output","and","the","entries","from","the","memory","bank",".","Entries","from","the","memory","bank","have","shape","(","dim",",","num_features",")","if","feature_dim_first","is","True","and","(","num_features",",","dim",")","otherwise","."]
code: def forward(
      self,
      output: Tensor,
      labels: Union[Tensor, None] = None,
      update: bool = False,
  ) -> Tuple[Tensor, Union[Tensor, None]]:
      """Query memory bank for additional negative samples
  
      Args:
          output:
              The output of the model.
          labels:
              Should always be None, will be ignored.
          update:
              If True, the memory bank will be updated with the current output.
  
      Returns:
          The output if the memory bank is of size 0, otherwise the output
          and the entries from the memory bank. Entries from the memory bank have
          shape (dim, num_features) if feature_dim_first is True and
          (num_features, dim) otherwise.
  
      """
  
      # no memory bank, return the output
      if self.size[0] == 0:
          return output, None
  
      # Initialize the memory bank if it is not already done.
      if self.bank.ndim == 1:
          dim = output.shape[1:]
          self._init_memory_bank(size=(*self.size, *dim))
  
      # query and update memory bank
      bank = self.bank.clone().detach()
      if self.feature_dim_first:
          # swap bank size and feature dimension for backwards compatibility
          bank = bank.transpose(0, -1)
  
      # only update memory bank if we later do backward pass (gradient)
      if update:
          self._dequeue_and_enqueue(output)
  
      return output, bank
  
file_path: lightly/lightly/models/modules/memory_bank.py
imports: import warnings
  from typing import Sequence, Tuple, Union
  import torch
  from torch import Tensor
  from torch.nn import Module
  from lightly.models import utils
label_desc: Use image node_id 4 for calling the MemoryBankModule obj's underlying member method code with example usage: obj.forward(output, labels, update) and returns: output,  bank, output,  None