class_docstr: Loss scale that dynamically adjusts itself.
  
  Dynamic loss scaling works by adjusting the loss scale as training progresses.
  The goal is to keep the loss scale as high as possible without overflowing the
  gradients. As long as the gradients do not overflow, raising the loss scale
  never hurts.
  
  The algorithm starts by setting the loss scale to an initial value. Every N
  steps that the gradients are finite, the loss scale is increased by some
  factor. However, if a NaN or Inf gradient is found, the gradients for that
  step are not applied, and the loss scale is decreased by the factor. This
  process tends to keep the loss scale as high as possible without gradients
  overflowing.
class_docstr_tok: ["Loss","scale","that","dynamically","adjusts","itself",".","Dynamic","loss","scaling","works","by","adjusting","the","loss","scale","as","training","progresses",".","The","goal","is","to","keep","the","loss","scale","as","high","as","possible","without","overflowing","the","gradients",".","As","long","as","the","gradients","do","not","overflow",",","raising","the","loss","scale","never","hurts",".","The","algorithm","starts","by","setting","the","loss","scale","to","an","initial","value",".","Every","N","steps","that","the","gradients","are","finite",",","the","loss","scale","is","increased","by","some","factor",".","However",",","if","a","NaN","or","Inf","gradient","is","found",",","the","gradients","for","that","step","are","not","applied",",","and","the","loss","scale","is","decreased","by","the","factor",".","This","process","tends","to","keep","the","loss","scale","as","high","as","possible","without","gradients","overflowing","."]
docstr: Creates the dynamic loss scale.
  
  Args:
    initial_loss_scale: A Python float.  The loss scale to use at the
      beginning. It's better to start this at a very high number, because a
      loss scale that is too high gets lowered far more quickly than a loss
      scale that is too low gets raised. The default is 2 ** 15, which is
      approximately half the maximum float16 value.
    increment_period: Increases loss scale every `increment_period`
      consecutive steps that finite gradients are encountered. If a nonfinite
      gradient is encountered, the count is reset back to zero.
    multiplier: The multiplier to use when increasing or decreasing the loss
      scale.
docstr_tok: ["Creates","the","dynamic","loss","scale",".","Args",":","initial_loss_scale",":","A","Python","float",".","The","loss","scale","to","use","at","the","beginning",".","It","'s","better","to","start","this","at","a","very","high","number",",","because","a","loss","scale","that","is","too","high","gets","lowered","far","more","quickly","than","a","loss","scale","that","is","too","low","gets","raised",".","The","default","is","2","*","*","15",",","which","is","approximately","half","the","maximum","float16","value",".","increment_period",":","Increases","loss","scale","every","`","increment_period","`","consecutive","steps","that","finite","gradients","are","encountered",".","If","a","nonfinite","gradient","is","encountered",",","the","count","is","reset","back","to","zero",".","multiplier",":","The","multiplier","to","use","when","increasing","or","decreasing","the","loss","scale","."]
code: def __init__(
      self,
      initial_loss_scale=2**15,  # See docstring for why this is big.
      increment_period=2000,
      multiplier=2.0,
  ):
      """Creates the dynamic loss scale.
  
      Args:
        initial_loss_scale: A Python float.  The loss scale to use at the
          beginning. It's better to start this at a very high number, because a
          loss scale that is too high gets lowered far more quickly than a loss
          scale that is too low gets raised. The default is 2 ** 15, which is
          approximately half the maximum float16 value.
        increment_period: Increases loss scale every `increment_period`
          consecutive steps that finite gradients are encountered. If a nonfinite
          gradient is encountered, the count is reset back to zero.
        multiplier: The multiplier to use when increasing or decreasing the loss
          scale.
      """
      super(DynamicLossScale, self).__init__()
      self._initial_loss_scale = float(initial_loss_scale)
      self._increment_period = int(increment_period)
      self._multiplier = float(multiplier)
  
      self._current_loss_scale = self._add_weight(
          name="current_loss_scale",
          dtype=dtypes.float32,
          initial_value=self._initial_loss_scale,
      )
      # The number of consecutive steps with finite gradients since the last
      # nonfinite gradient or change in loss scale.
      self._num_good_steps = self._add_weight(
          name="good_steps", dtype=dtypes.int64, initial_value=0
      )
  
imports: import abc
  from tensorflow.python.distribute import distribute_lib
  from tensorflow.python.distribute import reduce_util
  from tensorflow.python.eager import context
  from tensorflow.python.framework import dtypes
  from tensorflow.python.framework import indexed_slices
  from tensorflow.python.framework import ops
  from tensorflow.python.ops import cond
  from tensorflow.python.ops import control_flow_ops
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import variable_v1
  from tensorflow.python.ops import variables
  from tensorflow.python.trackable import base
  from tensorflow.python.util import deprecation
  from tensorflow.python.util import nest
  from tensorflow.python.util.tf_export import tf_export
label_desc: Use image node_id 1 to create a new DynamicLossScale object from inherited base classes: LossScale with example: obj = DynamicLossScale(initial_loss_scale, increment_period, multiplier)