class_docstr: The AdamW optimizer.
  
  This class implements the optimizer described in *Decoupled Weight Decay Regularization*,
   available at https://arxiv.org/pdf/1711.05101.pdf.
  
  Updates are applied by::
  
      grad = clip(grad * rescale_grad, clip_gradient)
      m = beta1 * m + (1 - beta1) * grad
      v = beta2 * v + (1 - beta2) * (grad**2)
      lr = learning_rate * sqrt(1 - beta2**t) / (1 - beta1**t)
      w = w - lr * (m / (sqrt(v) + epsilon) + wd * w)
  
  
  Also, we can turn off the bias correction term and the updates are as follows::
  
      grad = clip(grad * rescale_grad, clip_gradient) + wd * weight
      m = beta1 * m + (1 - beta1) * grad
      v = beta2 * v + (1 - beta2) * (grad**2)
      lr = learning_rate
      w = w - lr * (m / (sqrt(v) + epsilon) + wd * w)
  
  This optimizer accepts the following parameters in addition to those accepted
  by :class:`.Optimizer`.
  
  
  Parameters
  ----------
  learning_rate : float, default 0.001
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  beta1 : float, default 0.9
      Exponential decay rate for the first moment estimates.
  beta2 : float, default 0.999
      Exponential decay rate for the second moment estimates.
  epsilon : float, default 1e-6
      Small value to avoid division by 0.
  correct_bias : bool, default True
     Can be set to False to avoid correcting bias in Adam (e.g. like in Bert TF repository).
     Default True.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
docstr: Perform an optimization step using gradients and states.
  
  Parameters
  ----------
  indices : list of int
      List of unique indices of the parameters into the individual learning rates
      and weight decays. Learning rates and weight decay may be set via `set_lr_mult()`
      and `set_wd_mult()`, respectively.
  weights : list of NDArray
      List of parameters to be updated.
  grads : list of NDArray
      List of gradients of the objective with respect to this parameter.
  states : List of any obj
      List of state returned by `create_state()`.
code: def step(self, indices, weights, grads, states):
      """Perform an optimization step using gradients and states.
  
      Parameters
      ----------
      indices : list of int
          List of unique indices of the parameters into the individual learning rates
          and weight decays. Learning rates and weight decay may be set via `set_lr_mult()`
          and `set_wd_mult()`, respectively.
      weights : list of NDArray
          List of parameters to be updated.
      grads : list of NDArray
          List of gradients of the objective with respect to this parameter.
      states : List of any obj
          List of state returned by `create_state()`.
      """
      for index, weight, grad, state in zip(
          indices, weights, grads, states
      ):
          self._update_count(index)
          lr = self._get_lr(index)
          wd = self._get_wd(index)
          t = self._index_update_count[index]
  
          # preprocess grad
          grad *= self.rescale_grad
          if self.clip_gradient is not None:
              grad = clip(grad, -self.clip_gradient, self.clip_gradient)
          if self.correct_bias:
              coef1 = 1.0 - self.beta1**t
              coef2 = 1.0 - self.beta2**t
              lr *= math.sqrt(coef2) / coef1
  
          # update mean and var
          mean, var = state
          mean[:] *= self.beta1
          mean[:] += (1.0 - self.beta1) * grad
          var[:] *= self.beta2
          var[:] += (1.0 - self.beta2) * square(grad)
  
          # update weight
          d = mean / (sqrt(var) + self.epsilon)
          weight[:] -= lr * d
          # add wd
          if wd > 0:
              weight[:] -= lr * wd * weight
  
file_path: mxnet/python/mxnet/optimizer/adamW.py
imports: import math
  import os
  import numpy
  from .optimizer import Optimizer, register
  from ..ndarray import zeros, clip, sqrt, square, full, NDArray
  from ..ndarray.contrib import mp_adamw_update, adamw_update, multi_mp_adamw_update, multi_adamw_update
label_desc: Use image node_id 3 for calling the AdamW obj's underlying member method code with example usage: obj.step(indices, weights, grads, states) without return types