class_docstr: A wrapper over pytorch GradScaler made specifically for training hivemind.Optimizer with reuse_grad_buffers=True.
  
  :note: if not using reuse_grad_buffers=True, one can and *should* train normally without this class, e.g. using
    standard PyTorch AMP or Apex. This custom GradScaler is more memory-efficient, but requires custom training code.
  
  hivemind.GradScaler makes 3 modifications to the regular PyTorch AMP:
  
  - bypass .unscale_ and .update calls in order to accumulate gradients over several steps
  - limit increasing gradient scale to only immediately after global optimizer steps
  - allow training with some or master parameters in float16
  
  :note: The above modiffications will be enabled automatically. One can (and should) use hivemind.GradScaler exactly
    as regular ``torch.amp.GradScaler``.
code: def update(self, new_scale: Optional[float] = None) -> bool:
      with self._lock:
          total_infs = 0
          for optimizer_state in self._per_optimizer_states.values():
              total_infs += sum(
                  v.item()
                  for v in optimizer_state[
                      "found_inf_per_device"
                  ].values()
              )
  
          if self._is_ready_to_update or total_infs != 0:
              # note: we update either during actual optimizer step or if we need to reduce scale due to NaN
              super().update(new_scale)
              self._is_ready_to_update = False
              return True
          else:
              for opt_id in self._optimizer_states_to_reset:
                  self._per_optimizer_states[
                      opt_id
                  ] = _refresh_per_optimizer_state()
              self._optimizer_states_to_reset.clear()
              return False
  
imports: import contextlib
  import threading
  from copy import deepcopy
  from typing import Dict, Optional
  import torch
  from torch.cuda.amp import GradScaler
  from torch.cuda.amp.grad_scaler import OptState, _refresh_per_optimizer_state
  from torch.optim import Optimizer
  import hivemind
  from hivemind.utils.logging import get_logger
label_desc: Use image node_id 5 for calling the GradScaler obj's underlying member method code with example usage: obj.update(new_scale) and returns: True, False