class_docstr: A wrapper over pytorch GradScaler made specifically for training hivemind.Optimizer with reuse_grad_buffers=True.
  
  :note: if not using reuse_grad_buffers=True, one can and *should* train normally without this class, e.g. using
    standard PyTorch AMP or Apex. This custom GradScaler is more memory-efficient, but requires custom training code.
  
  hivemind.GradScaler makes 3 modifications to the regular PyTorch AMP:
  
  - bypass .unscale_ and .update calls in order to accumulate gradients over several steps
  - limit increasing gradient scale to only immediately after global optimizer steps
  - allow training with some or master parameters in float16
  
  :note: The above modiffications will be enabled automatically. One can (and should) use hivemind.GradScaler exactly
    as regular ``torch.amp.GradScaler``.
code: def unscale_(self, optimizer: TorchOptimizer) -> bool:
      with self._lock:
          assert isinstance(optimizer, hivemind.Optimizer)
          if self._is_running_global_step:
              super().unscale_(optimizer)
              self._inner_optimizer_states[
                  id(optimizer.opt)
              ] = deepcopy(self._per_optimizer_states[id(optimizer)])
              # note: we store unscaled optimizer state in a separate dict and not in _per_optimizer_states in order
              # to avoid an edge case where full DPU peer encounters overflow in local gradients while averaging
              # offloaded gradients (i.e. after global unscale but before global step). Due to overflow, next call to
              # .update on user side would reset *all* optimizer states and cause .step to unscale gradients twice.
              # Offloaded optimizer is not affected by overflow in on-device gradients and should not be reset.
              return True
          else:
              self._check_inf_per_device(optimizer)
              self._optimizer_states_to_reset.add(id(optimizer))
              return False
  
imports: import contextlib
  import threading
  from copy import deepcopy
  from typing import Dict, Optional
  import torch
  from torch.cuda.amp import GradScaler
  from torch.cuda.amp.grad_scaler import OptState, _refresh_per_optimizer_state
  from torch.optim import Optimizer
  import hivemind
  from hivemind.utils.logging import get_logger
label_desc: Use image node_id 3 for calling the GradScaler obj's underlying member method code with example usage: obj.unscale_(optimizer) and returns: True, False