class_docstr: Boosted version of MultipleNegativesRankingLoss (https://arxiv.org/pdf/1705.00652.pdf) by GradCache (https://arxiv.org/pdf/2101.06983.pdf).
  
  Constrastive learning (here our MNRL loss) with in-batch negatives is usually hard to work with large batch sizes due to (GPU) memory limitation.
  Even with batch-scaling methods like gradient-scaling, it cannot work either. This is because the in-batch negatives make the data points within
  the same batch non-independent and thus the batch cannot be broke down into mini-batches. GradCache is a smart way to solve this problem.
  It achieves the goal by dividing the computation into two stages of embedding and loss calculation, which both can be scaled by mini-batches.
  As a result, memory of constant size (e.g. that works with batch size = 32) can now process much larger batches (e.g. 65536).
  
  In detail:
      (1) It first does a quick embedding step without gradients/computation graphs to get all the embeddings;
      (2) Calculate the loss, backward up to the embeddings and cache the gradients wrt. to the embeddings;
      (3) A 2nd embedding step with gradients/computation graphs and connect the cached gradients into the backward chain.
  
  Notes: All steps are done with mini-batches. In the original implementation of GradCache, (2) is not done in mini-batches and
  requires a lot memory when batch size large. One drawback is about the speed. GradCache will sacrifice around 20% computation time according to the paper.
  
  Example:
  
      from sentence_transformers import SentenceTransformer
      train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=1024)  # Here we can try much larger batch sizes!
      train_loss = losses.CachedMultipleNegativesRankingLoss(model=model, mini_batch_size: int = 32)
docstr: :param model: SentenceTransformer model
  :param scale: Output of similarity function is multiplied by scale value
  :param similarity_fct: similarity function between sentence embeddings. By default, cos_sim. Can also be set to dot product (and then set scale to 1)
docstr_tok: [":","param","model",":","SentenceTransformer","model",":","param","scale",":","Output","of","similarity","function","is","multiplied","by","scale","value",":","param","similarity_fct",":","similarity","function","between","sentence","embeddings",".","By","default",",","cos_sim",".","Can","also","be","set","to","dot","product","(","and","then","set","scale","to","1",")"]
code: def __init__(
      self,
      model: SentenceTransformer,
      scale: float = 20.0,
      similarity_fct: callable[[Tensor, Tensor], Tensor] = util.cos_sim,
      mini_batch_size: int = 32,
      show_progress_bar: bool = False,
  ):
      """
      :param model: SentenceTransformer model
      :param scale: Output of similarity function is multiplied by scale value
      :param similarity_fct: similarity function between sentence embeddings. By default, cos_sim. Can also be set to dot product (and then set scale to 1)
      """
      super(CachedMultipleNegativesRankingLoss, self).__init__()
      self.model = model
      self.scale = scale
      self.similarity_fct = similarity_fct
      self.cross_entropy_loss = nn.CrossEntropyLoss()
      self.mini_batch_size = mini_batch_size
      self.cache: Optional[List[List[Tensor]]] = None
      self.random_states: Optional[List[List[RandContext]]] = None
      self.show_progress_bar = show_progress_bar
  
imports: from __future__ import annotations
  from contextlib import nullcontext
  from functools import partial
  import torch
  from torch import nn, Tensor
  from torch.utils.checkpoint import get_device_states, set_device_states
  from typing import Iterable, Dict, Iterator, List, Optional, Tuple
  from sentence_transformers import SentenceTransformer
  from sentence_transformers import util
  import tqdm
label_desc: Use image node_id 1 to create a new CachedMultipleNegativesRankingLoss object from inherited base classes: nn with example: obj = CachedMultipleNegativesRankingLoss(model, scale, similarity_fct, mini_batch_size, show_progress_bar)