class_docstr: Optimizer that implements the Momentum algorithm with weight_decay.
  
  This is an implementation of the SGDW optimizer described in "Decoupled
  Weight Decay Regularization" by [Loshchilov & Hutter]
  (https://arxiv.org/pdf/1711.05101.pdf).
  It computes the update step of `tf.keras.optimizers.SGD` and additionally
  decays the variable. Note that this is different from adding
  L2 regularization on the variables to the loss. Decoupling the weight decay
  from other hyperparameters (in particular the learning rate) simplifies
  hyperparameter search.
  
  For further information see the documentation of the SGD Optimizer.
  
  This optimizer can also be instantiated as
  ```python
  extend_with_decoupled_weight_decay(tf.keras.optimizers.SGD,
                                     weight_decay=weight_decay)
  ```
  
  Note: when applying a decay to the learning rate, be sure to manually apply
  the decay to the `weight_decay` as well. For example:
  
  ```python
  step = tf.Variable(0, trainable=False)
  schedule = tf.optimizers.schedules.PiecewiseConstantDecay(
      [10000, 15000], [1e-0, 1e-1, 1e-2])
  # lr and wd can be a function or a tensor
  lr = 1e-1 * schedule(step)
  wd = lambda: 1e-4 * schedule(step)
  
  # ...
  
  optimizer = tfa.optimizers.SGDW(
      learning_rate=lr, weight_decay=wd, momentum=0.9)
  ```
docstr: Construct a new SGDW optimizer.
  
  For further information see the documentation of the SGD Optimizer.
  
  Args:
      learning_rate: float hyperparameter >= 0. Learning rate.
      momentum: float hyperparameter >= 0 that accelerates SGD in the
          relevant direction and dampens oscillations.
      nesterov: boolean. Whether to apply Nesterov momentum.
      name: Optional name prefix for the operations created when applying
          gradients.  Defaults to 'SGD'.
      **kwargs: keyword arguments. Allowed to be {`clipnorm`, `clipvalue`,
          `lr`, `decay`, `exclude_from_weight_decay`}. `clipnorm` is clip
          gradients by norm; `clipvalue` is clip gradients by value.
          `decay` is included for backward compatibility to allow time
          inverse decay of learning rate. `lr` is included for backward
          compatibility, recommended to use `learning_rate` instead.
          `exclude_from_weight_decay` accepts list of regex patterns of
          variables excluded from weight decay.
docstr_tok: ["Construct","a","new","SGDW","optimizer",".","For","further","information","see","the","documentation","of","the","SGD","Optimizer",".","Args",":","learning_rate",":","float","hyperparameter",">","=","0",".","Learning","rate",".","momentum",":","float","hyperparameter",">","=","0","that","accelerates","SGD","in","the","relevant","direction","and","dampens","oscillations",".","nesterov",":","boolean",".","Whether","to","apply","Nesterov","momentum",".","name",":","Optional","name","prefix","for","the","operations","created","when","applying","gradients",".","Defaults","to","'SGD","'",".","*","*","kwargs",":","keyword","arguments",".","Allowed","to","be","{","`","clipnorm","`",",","`","clipvalue","`",",","`","lr","`",",","`","decay","`",",","`","exclude_from_weight_decay","`","}",".","`","clipnorm","`","is","clip","gradients","by","norm",";","`","clipvalue","`","is","clip","gradients","by","value",".","`","decay","`","is","included","for","backward","compatibility","to","allow","time","inverse","decay","of","learning","rate",".","`","lr","`","is","included","for","backward","compatibility",",","recommended","to","use","`","learning_rate","`","instead",".","`","exclude_from_weight_decay","`","accepts","list","of","regex","patterns","of","variables","excluded","from","weight","decay","."]
code: def __init__(
      self,
      weight_decay: Union[FloatTensorLike, Callable],
      learning_rate: Union[FloatTensorLike, Callable] = 0.001,
      momentum: Union[FloatTensorLike, Callable] = 0.0,
      nesterov: bool = False,
      name: str = "SGDW",
      **kwargs,
  ):
      """Construct a new SGDW optimizer.
  
      For further information see the documentation of the SGD Optimizer.
  
      Args:
          learning_rate: float hyperparameter >= 0. Learning rate.
          momentum: float hyperparameter >= 0 that accelerates SGD in the
              relevant direction and dampens oscillations.
          nesterov: boolean. Whether to apply Nesterov momentum.
          name: Optional name prefix for the operations created when applying
              gradients.  Defaults to 'SGD'.
          **kwargs: keyword arguments. Allowed to be {`clipnorm`, `clipvalue`,
              `lr`, `decay`, `exclude_from_weight_decay`}. `clipnorm` is clip
              gradients by norm; `clipvalue` is clip gradients by value.
              `decay` is included for backward compatibility to allow time
              inverse decay of learning rate. `lr` is included for backward
              compatibility, recommended to use `learning_rate` instead.
              `exclude_from_weight_decay` accepts list of regex patterns of
              variables excluded from weight decay.
      """
      super().__init__(
          weight_decay,
          learning_rate=learning_rate,
          momentum=momentum,
          nesterov=nesterov,
          name=name,
          **kwargs,
      )
  
imports: import importlib
  import tensorflow
  from tensorflow_addons.utils.types import FloatTensorLike
  from tensorflow_addons.optimizers.utils import is_variable_matched_by_regexes
  from typeguard import typechecked
  from typing import Union, Callable, Type, Optional, List
label_desc: Use image node_id 1 to create a new SGDW object from inherited base classes: DecoupledWeightDecayExtension, SGD_CLASS with example: obj = SGDW(weight_decay, learning_rate, momentum, nesterov, name)