class_docstr: A wrapper over pytorch GradScaler made specifically for training hivemind.Optimizer with reuse_grad_buffers=True.
  
  :note: if not using reuse_grad_buffers=True, one can and *should* train normally without this class, e.g. using
    standard PyTorch AMP or Apex. This custom GradScaler is more memory-efficient, but requires custom training code.
  
  hivemind.GradScaler makes 3 modifications to the regular PyTorch AMP:
  
  - bypass .unscale_ and .update calls in order to accumulate gradients over several steps
  - limit increasing gradient scale to only immediately after global optimizer steps
  - allow training with some or master parameters in float16
  
  :note: The above modiffications will be enabled automatically. One can (and should) use hivemind.GradScaler exactly
    as regular ``torch.amp.GradScaler``.
code: def _unscale_grads_(
      self,
      optimizer: TorchOptimizer,
      inv_scale: torch.Tensor,
      found_inf: torch.Tensor,
      allow_fp16: bool,
  ) -> Dict[torch.device, torch.Tensor]:
      # note: the code below sets allow_fp16=True to allow training with master weights (partially) in fp16
      # inspired by: https://github.com/facebookresearch/fairscale/blob/945b9666/fairscale/optim/grad_scaler.py
      return super()._unscale_grads_(
          optimizer, inv_scale, found_inf, allow_fp16=True
      )
  
imports: import contextlib
  import threading
  from copy import deepcopy
  from typing import Dict, Optional
  import torch
  from torch.cuda.amp import GradScaler
  from torch.cuda.amp.grad_scaler import OptState, _refresh_per_optimizer_state
  from torch.optim import Optimizer
  import hivemind
  from hivemind.utils.logging import get_logger
label_desc: Use image node_id 6 for calling the GradScaler obj's underlying member method code with example usage: obj._unscale_grads_(optimizer, inv_scale, found_inf, allow_fp16) and returns: super