class_docstr: Dynamically adjusts the loss scaling factor.
  
  Dynamic loss scalers are important in mixed-precision training. They help
  us avoid underflows and overflows in low-precision gradients.
  
  See here for information:
  <https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html#lossscaling>
  
  Shamelessly stolen and adapted from Fairseq.
  <https://github.com/pytorch/fairseq/blob/main/fairseq/optim/fp16_optimizer.py>
class_docstr_tok: ["Dynamically","adjusts","the","loss","scaling","factor",".","Dynamic","loss","scalers","are","important","in","mixed-precision","training",".","They","help","us","avoid","underflows","and","overflows","in","low-precision","gradients",".","See","here","for","information",":","<","https",":","\/\/docs.nvidia.com\/deeplearning\/performance\/mixed-precision-training\/index.html","#","lossscaling",">","Shamelessly","stolen","and","adapted","from","Fairseq",".","<","https",":","\/\/github.com\/pytorch\/fairseq\/blob\/main\/fairseq\/optim\/fp16_optimizer.py",">"]
docstr: Update the loss scale.
  
  If overflow exceeds our tolerance, we decrease the loss scale. If the number of
  iterations since the last overflow exceeds the scale window, we increase the
  loss scale.
code: def update_scale(self, overflow: bool):
      """
      Update the loss scale.
  
      If overflow exceeds our tolerance, we decrease the loss scale. If the number of
      iterations since the last overflow exceeds the scale window, we increase the
      loss scale.
      """
      iter_since_rescale = self._iter - self._last_rescale_iter
  
      if overflow:
          # calculate how often we overflowed already
          self._last_overflow_iter = self._iter
          self._overflows_since_rescale += 1
          pct_overflow = self._overflows_since_rescale / float(
              iter_since_rescale
          )
          if pct_overflow >= self.tolerance:
              # decrease loss scale by the scale factor
              self._decrease_loss_scale()
              # reset iters
              self._last_rescale_iter = self._iter
              self._overflows_since_rescale = 0
      elif (
          self._iter - self._last_overflow_iter
      ) % self.scale_window == 0:
          # increase the loss scale by scale factor
          self.loss_scale *= self.scale_factor
          self._last_rescale_iter = self._iter
  
      self._iter += 1
  
imports: import math
  from itertools import chain
  from typing import Optional
  import parlai.utils.logging
label_desc: Use image node_id 2 for calling the DynamicLossScaler obj's underlying member method code with example usage: obj.update_scale(overflow) without return types