class_docstr: Optimizer that computes a moving average of the variables.
  
  Empirically it has been found that using the moving average of the trained
  parameters of a deep network is better than using its trained parameters
  directly. This optimizer allows you to compute this moving average and swap
  the variables at save time so that any code outside of the training loop
  will use by default the average values instead of the original ones.
  
  Example of usage:
  
  ```python
  opt = tf.keras.optimizers.SGD(learning_rate)
  opt = tfa.optimizers.MovingAverage(opt)
  
  ```
class_docstr_tok: ["Optimizer","that","computes","a","moving","average","of","the","variables",".","Empirically","it","has","been","found","that","using","the","moving","average","of","the","trained","parameters","of","a","deep","network","is","better","than","using","its","trained","parameters","directly",".","This","optimizer","allows","you","to","compute","this","moving","average","and","swap","the","variables","at","save","time","so","that","any","code","outside","of","the","training","loop","will","use","by","default","the","average","values","instead","of","the","original","ones",".","Example","of","usage",":","``","`","python","opt","=","tf.keras.optimizers.SGD","(","learning_rate",")","opt","=","tfa.optimizers.MovingAverage","(","opt",")","``","`"]
docstr: Construct a new MovingAverage optimizer.
  
  Args:
      optimizer: str or `tf.keras.optimizers.legacy.Optimizer` that will be
          used to compute and apply gradients.
      average_decay: float. Decay to use to maintain the moving averages
          of trained variables.
      num_updates: Optional count of the number of updates applied to
          variables.
      start_step: int. What step to start the moving average.
      dynamic_decay: bool. Whether to change the decay based on the number
          of optimizer updates. Decay will start at 0.1 and gradually
          increase up to `average_decay` after each optimizer update.
      name: Optional name for the operations created when applying
          gradients. Defaults to "MovingAverage".
      **kwargs: keyword arguments. Allowed to be {`clipnorm`,
          `clipvalue`, `lr`, `decay`}. `clipnorm` is clip gradients by
          norm; `clipvalue` is clip gradients by value, `decay` is
          included for backward compatibility to allow time inverse
          decay of learning rate. `lr` is included for backward
          compatibility, recommended to use `learning_rate` instead.
docstr_tok: ["Construct","a","new","MovingAverage","optimizer",".","Args",":","optimizer",":","str","or","`","tf.keras.optimizers.legacy.Optimizer","`","that","will","be","used","to","compute","and","apply","gradients",".","average_decay",":","float",".","Decay","to","use","to","maintain","the","moving","averages","of","trained","variables",".","num_updates",":","Optional","count","of","the","number","of","updates","applied","to","variables",".","start_step",":","int",".","What","step","to","start","the","moving","average",".","dynamic_decay",":","bool",".","Whether","to","change","the","decay","based","on","the","number","of","optimizer","updates",".","Decay","will","start","at","0.1","and","gradually","increase","up","to","`","average_decay","`","after","each","optimizer","update",".","name",":","Optional","name","for","the","operations","created","when","applying","gradients",".","Defaults","to","``","MovingAverage","''",".","*","*","kwargs",":","keyword","arguments",".","Allowed","to","be","{","`","clipnorm","`",",","`","clipvalue","`",",","`","lr","`",",","`","decay","`","}",".","`","clipnorm","`","is","clip","gradients","by","norm",";","`","clipvalue","`","is","clip","gradients","by","value",",","`","decay","`","is","included","for","backward","compatibility","to","allow","time","inverse","decay","of","learning","rate",".","`","lr","`","is","included","for","backward","compatibility",",","recommended","to","use","`","learning_rate","`","instead","."]
code: def __init__(
      self,
      optimizer: types.Optimizer,
      average_decay: types.FloatTensorLike = 0.99,
      num_updates: Union[None, int, tf.Variable] = None,
      start_step: int = 0,
      dynamic_decay: bool = False,
      name: str = "MovingAverage",
      **kwargs,
  ):
      r"""Construct a new MovingAverage optimizer.
  
      Args:
          optimizer: str or `tf.keras.optimizers.legacy.Optimizer` that will be
              used to compute and apply gradients.
          average_decay: float. Decay to use to maintain the moving averages
              of trained variables.
          num_updates: Optional count of the number of updates applied to
              variables.
          start_step: int. What step to start the moving average.
          dynamic_decay: bool. Whether to change the decay based on the number
              of optimizer updates. Decay will start at 0.1 and gradually
              increase up to `average_decay` after each optimizer update.
          name: Optional name for the operations created when applying
              gradients. Defaults to "MovingAverage".
          **kwargs: keyword arguments. Allowed to be {`clipnorm`,
              `clipvalue`, `lr`, `decay`}. `clipnorm` is clip gradients by
              norm; `clipvalue` is clip gradients by value, `decay` is
              included for backward compatibility to allow time inverse
              decay of learning rate. `lr` is included for backward
              compatibility, recommended to use `learning_rate` instead.
      """
      super().__init__(optimizer, name, **kwargs)
      self._num_updates = num_updates
      if self._num_updates is not None:
          if isinstance(self._num_updates, tf.Variable):
              tf.debugging.assert_integer(
                  self._num_updates,
                  (
                      'type of argument "num_updates" must be '
                      "int; got {} instead".format(
                          self._num_updates.dtype
                      )
                  ),
              )
          num_updates = tf.cast(
              self._num_updates, tf.float32, name="num_updates"
          )
          average_decay = tf.minimum(
              average_decay, (1.0 + num_updates) / (10.0 + num_updates)
          )
  
      self._set_hyper("average_decay", average_decay)
      self._start_step = start_step
      self._dynamic_decay = dynamic_decay
  
file_path: addons/tensorflow_addons/optimizers/moving_average.py
imports: import tensorflow
  from tensorflow_addons.optimizers import AveragedOptimizerWrapper
  from tensorflow_addons.utils import types
  from typing import Union
  from typeguard import typechecked
label_desc: Use image node_id 1 to create a new MovingAverage object from inherited base classes: AveragedOptimizerWrapper with example: obj = MovingAverage(optimizer, average_decay, num_updates, start_step, dynamic_decay, name)