class_docstr: Memory bank implementation
  
  This is a parent class to all loss functions implemented by the lightly
  Python package. This way, any loss can be used with a memory bank if
  desired.
  
  Attributes:
      size:
          Size of the memory bank as (num_features, dim) tuple. If num_features is 0
          then the memory bank is disabled. Deprecated: If only a single integer is
          passed, it is interpreted as the number of features and the feature
          dimension is inferred from the first batch stored in the memory bank.
          Leaving out the feature dimension might lead to errors in distributed
          training.
      gather_distributed:
          If True then negatives from all gpus are gathered before the memory bank
          is updated. This results in more frequent updates of the memory bank and
          keeps the memory bank contents independent of the number of gpus. But it has
          the drawback that synchronization between processes is required and
          diversity of the memory bank content is reduced.
      feature_dim_first:
          If True, the memory bank returns features with shape (dim, num_features).
          If False, the memory bank returns features with shape (num_features, dim).
  
  Examples:
      >>> class MyLossFunction(MemoryBankModule):
      >>>
      >>>     def __init__(self, memory_bank_size: Tuple[int, int] = (2 ** 16, 128)):
      >>>         super().__init__(memory_bank_size)
      >>>
      >>>     def forward(self, output: Tensor, labels: Union[Tensor, None] = None):
      >>>         output, negatives = super().forward(output)
      >>>
      >>>         if negatives is not None:
      >>>             # evaluate loss with negative samples
      >>>         else:
      >>>             # evaluate loss without negative samples
class_docstr_tok: ["Memory","bank","implementation","This","is","a","parent","class","to","all","loss","functions","implemented","by","the","lightly","Python","package",".","This","way",",","any","loss","can","be","used","with","a","memory","bank","if","desired",".","Attributes",":","size",":","Size","of","the","memory","bank","as","(","num_features",",","dim",")","tuple",".","If","num_features","is","0","then","the","memory","bank","is","disabled",".","Deprecated",":","If","only","a","single","integer","is","passed",",","it","is","interpreted","as","the","number","of","features","and","the","feature","dimension","is","inferred","from","the","first","batch","stored","in","the","memory","bank",".","Leaving","out","the","feature","dimension","might","lead","to","errors","in","distributed","training",".","gather_distributed",":","If","True","then","negatives","from","all","gpus","are","gathered","before","the","memory","bank","is","updated",".","This","results","in","more","frequent","updates","of","the","memory","bank","and","keeps","the","memory","bank","contents","independent","of","the","number","of","gpus",".","But","it","has","the","drawback","that","synchronization","between","processes","is","required","and","diversity","of","the","memory","bank","content","is","reduced",".","feature_dim_first",":","If","True",",","the","memory","bank","returns","features","with","shape","(","dim",",","num_features",")",".","If","False",",","the","memory","bank","returns","features","with","shape","(","num_features",",","dim",")",".","Examples",":",">",">",">","class","MyLossFunction","(","MemoryBankModule",")",":",">",">",">",">",">",">","def","__init__","(","self",",","memory_bank_size",":","Tuple","[","int",",","int","]","=","(","2","*","*","16",",","128",")",")",":",">",">",">","super","(",")",".__init__","(","memory_bank_size",")",">",">",">",">",">",">","def","forward","(","self",",","output",":","Tensor",",","labels",":","Union","[","Tensor",",","None","]","=","None",")",":",">",">",">","output",",","negatives","=","super","(",")",".forward","(","output",")",">",">",">",">",">",">","if","negatives","is","not","None",":",">",">",">","#","evaluate","loss","with","negative","samples",">",">",">","else",":",">",">",">","#","evaluate","loss","without","negative","samples"]
docstr: Initialize the memory bank.
  
  Args:
      size:
          Size of the memory bank as (num_features, dim) tuple.
docstr_tok: ["Initialize","the","memory","bank",".","Args",":","size",":","Size","of","the","memory","bank","as","(","num_features",",","dim",")","tuple","."]
code: def _init_memory_bank(self, size: Tuple[int, ...]) -> None:
      """Initialize the memory bank.
  
      Args:
          size:
              Size of the memory bank as (num_features, dim) tuple.
  
      """
      self.bank = torch.randn(size).type_as(self.bank)
      self.bank = torch.nn.functional.normalize(self.bank, dim=-1)
      self.bank_ptr = torch.zeros(1).type_as(self.bank_ptr)
  
file_path: lightly/lightly/models/modules/memory_bank.py
imports: import warnings
  from typing import Sequence, Tuple, Union
  import torch
  from torch import Tensor
  from torch.nn import Module
  from lightly.models import utils
label_desc: Use image node_id 2 for calling the MemoryBankModule obj's underlying member method code with example usage: obj._init_memory_bank(size) without return types