class_docstr: Gradient scaler for the PyTorch shim.
  
  Gradients with small magnitudes are not representable in half-precision and
  will underflow to zero. A gradient scaler counters this issue by scaling
  up the loss before backpropagation, increasing the gradients by the same
  magnitude. A large enough scale will avoid that the gradients underflow.
  The gradients are unscaled in single precision after backpropagation, to
  provide the unscaled gradients to the optimizer.
docstr: Construct a gradient scaler for the PyTorch shim.
  
  enabled (bool):
      Sets whether the gradient scalar is enabled. If it is disabled, the
      methods of the grad scaler are no-ops.
  
  init_scale (float):
      The initial scale used to increase the gradient magnitude.
  
  backoff_factor (float):
      The scale will be multiplied by this factor if any of the gradients
      overflows.
  
  growth_factor (float):
      The scale will be multiplied by this factor when none of the gradients
      overflowed for "growth_interval" steps.
  
  growth_interval (int):
      When no overflows were found for this number of steps, the scale will
      be multiplied by "growth_factor".
code: def __init__(
      self,
      enabled: bool = False,
      init_scale: float = 2.0**16,
      backoff_factor: float = 0.5,
      growth_factor: float = 2.0,
      growth_interval: int = 2000,
  ):
      """
      Construct a gradient scaler for the PyTorch shim.
  
      enabled (bool):
          Sets whether the gradient scalar is enabled. If it is disabled, the
          methods of the grad scaler are no-ops.
  
      init_scale (float):
          The initial scale used to increase the gradient magnitude.
  
      backoff_factor (float):
          The scale will be multiplied by this factor if any of the gradients
          overflows.
  
      growth_factor (float):
          The scale will be multiplied by this factor when none of the gradients
          overflowed for "growth_interval" steps.
  
      growth_interval (int):
          When no overflows were found for this number of steps, the scale will
          be multiplied by "growth_factor".
      """
      self._enabled = enabled
      self._growth_factor = growth_factor
      self._backoff_factor = backoff_factor
      self._growth_interval = growth_interval
  
      self._growth_tracker = torch.full((1,), 0, dtype=torch.int)
      self._scale = torch.full((1,), init_scale)
      self._found_inf = False
  
file_path: thinc/thinc/shims/pytorch_grad_scaler.py
imports: from typing import Dict, Iterable, List, Union, cast
  from ..compat import has_torch_amp, torch
  from ..util import is_torch_array
label_desc: Use image node_id 1 to create a new PyTorchGradScaler object  with example: obj = PyTorchGradScaler(enabled, init_scale, backoff_factor, growth_factor, growth_interval)