class_docstr: Class that manages dynamic loss scaling.  It is recommended to use :class:`DynamicLossScaler`
  indirectly, by supplying ``dynamic_loss_scale=True`` to the constructor of
  :class:`FP16_Optimizer`.  However, it's important to understand how :class:`DynamicLossScaler`
  operates, because the default options can be changed using the
  the ``dynamic_loss_args`` argument to :class:`FP16_Optimizer`'s constructor.
  
  Loss scaling is designed to combat the problem of underflowing gradients encountered at long
  times when training fp16 networks.  Dynamic loss scaling begins by attempting a very high loss
  scale.  Ironically, this may result in OVERflowing gradients.  If overflowing gradients are
  encountered, :class:`DynamicLossScaler` informs :class:`FP16_Optimizer` that an overflow has
  occurred.
  :class:`FP16_Optimizer` then skips the update step for this particular iteration/minibatch,
  and :class:`DynamicLossScaler` adjusts the loss scale to a lower value.
  If a certain number of iterations occur without overflowing gradients detected,
  :class:`DynamicLossScaler` increases the loss scale once more.
  In this way :class:`DynamicLossScaler` attempts to "ride the edge" of
  always using the highest loss scale possible without incurring overflow.
  
  Args:
      init_scale (float, optional, default=2**32):  Initial loss scale attempted by :class:`DynamicLossScaler.`
      scale_factor (float, optional, default=2.0):  Factor used when adjusting the loss scale. If an overflow is encountered, the loss scale is readjusted to loss scale/``scale_factor``.  If ``scale_window`` consecutive iterations take place without an overflow, the loss scale is readjusted to loss_scale*``scale_factor``.
      scale_window (int, optional, default=1000):  Number of consecutive iterations without an overflow to wait before increasing the loss scale.
      consecutive_hysteresis (bool, optional, default=False): Whether to refill hysteresis if we reach an iteration that doesn't overflow
code: def update_scale(self, overflow):
      if overflow:
          # self.cur_scale /= self.scale_factor
          if self.delayed_shift == 1 or self.cur_hysteresis == 1:
              if (
                  self.cur_scale == self.min_scale
              ) and self.raise_error_at_min_scale:
                  raise Exception(
                      "Current loss scale already at minimum - cannot decrease scale anymore. Exiting run."
                  )
              else:
                  next_scale = max(
                      self.cur_scale / self.scale_factor, self.min_scale
                  )
                  if dist.get_rank() == 0:
                      overflow_msg = f"[deepspeed] OVERFLOW! Rank {dist.get_rank()} Skipping step."
                      if self.dtype == torch.half:
                          overflow_msg += f" Attempted loss scale: {int(self.cur_scale)}, reducing to {int(next_scale)}"
                      logger.info(overflow_msg)
                  self.cur_scale = next_scale
          else:
              if dist.get_rank() == 0:
                  overflow_msg = f"[deepspeed] OVERFLOW! Rank {dist.get_rank()} Skipping step."
                  if self.dtype == torch.half:
                      overflow_msg += f" Attempted loss scale: {int(self.cur_scale)}, but hysteresis is {self.cur_hysteresis}. Reducing hysteresis to {self.cur_hysteresis-1}"
                  logger.info(overflow_msg)
              self.cur_hysteresis -= 1
          self.last_overflow_iter = self.cur_iter
      else:
          if self.consecutive_hysteresis:
              if dist.get_rank() == 0:
                  hysteresis_msg = f"Consecutive hysteresis is enabled. Restoring hysteresis to {self.delayed_shift}"
                  logger.info(hysteresis_msg)
              self.cur_hysteresis = self.delayed_shift
          if (
              self.cur_iter - self.last_overflow_iter
          ) % self.scale_window == 0:
              if not self.consecutive_hysteresis:
                  self.cur_hysteresis = self.delayed_shift
              self.cur_scale *= self.scale_factor
      self.cur_iter += 1
  
imports: import torch
  from deepspeed import comm
  from deepspeed.utils import logger
label_desc: Use image node_id 4 for calling the DynamicLossScaler obj's underlying member method code with example usage: obj.update_scale(overflow) without return types