class_docstr: Class that manages dynamic loss scaling.  It is recommended to use :class:`DynamicLossScaler`
  indirectly, by supplying ``dynamic_loss_scale=True`` to the constructor of
  :class:`FP16_Optimizer`.  However, it's important to understand how :class:`DynamicLossScaler`
  operates, because the default options can be changed using the
  the ``dynamic_loss_args`` argument to :class:`FP16_Optimizer`'s constructor.
  
  Loss scaling is designed to combat the problem of underflowing gradients encountered at long
  times when training fp16 networks.  Dynamic loss scaling begins by attempting a very high loss
  scale.  Ironically, this may result in OVERflowing gradients.  If overflowing gradients are
  encountered, :class:`DynamicLossScaler` informs :class:`FP16_Optimizer` that an overflow has
  occurred.
  :class:`FP16_Optimizer` then skips the update step for this particular iteration/minibatch,
  and :class:`DynamicLossScaler` adjusts the loss scale to a lower value.
  If a certain number of iterations occur without overflowing gradients detected,
  :class:`DynamicLossScaler` increases the loss scale once more.
  In this way :class:`DynamicLossScaler` attempts to "ride the edge" of
  always using the highest loss scale possible without incurring overflow.
  
  Args:
      init_scale (float, optional, default=2**32):  Initial loss scale attempted by :class:`DynamicLossScaler.`
      scale_factor (float, optional, default=2.0):  Factor used when adjusting the loss scale. If an overflow is encountered, the loss scale is readjusted to loss scale/``scale_factor``.  If ``scale_window`` consecutive iterations take place without an overflow, the loss scale is readjusted to loss_scale*``scale_factor``.
      scale_window (int, optional, default=1000):  Number of consecutive iterations without an overflow to wait before increasing the loss scale.
      consecutive_hysteresis (bool, optional, default=False): Whether to refill hysteresis if we reach an iteration that doesn't overflow
code: def has_overflow_serial(self, params):
      for p in params:
          if p.grad is not None and self._has_inf_or_nan(p.grad.data):
              return True
  
      return False
  
imports: import torch
  from deepspeed import comm
  from deepspeed.utils import logger
label_desc: Use image node_id 2 for calling the DynamicLossScaler obj's underlying member method code with example usage: obj.has_overflow_serial(params) and returns: False, True