class_docstr: The AdaBelief optimizer.
  
  This class implements the optimizer described in *Adapting Stepsizes by the Belief in Observed Gradients*,
   available at https://arxiv.org/pdf/2010.07468.pdf.
  
  Updates are applied by::
  
      grad = clip(grad * rescale_grad, clip_gradient) + wd * w
      m = beta1 * m + (1 - beta1) * grad
      s = beta2 * s + (1 - beta2) * ((grad - m)**2) + epsilon
      lr = learning_rate * sqrt(1 - beta2**t) / (1 - beta1**t)
      w = w - lr * (m / (sqrt(s) + epsilon))
  
  
  Also, we can turn off the bias correction term and the updates are as follows::
  
      grad = clip(grad * rescale_grad, clip_gradient) + wd * w
      m = beta1 * m + (1 - beta1) * grad
      s = beta2 * s + (1 - beta2) * ((grad - m)**2) + epsilon
      lr = learning_rate
      w = w - lr * (m / (sqrt(s) + epsilon))
  
  This optimizer accepts the following parameters in addition to those accepted
  by :class:`.Optimizer`.
  
  Parameters
  ----------
  learning_rate : float, default 0.001
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  beta1 : float, default 0.9
      Exponential decay rate for the first moment estimates.
  beta2 : float, default 0.999
      Exponential decay rate for the second moment estimates.
  epsilon : float, default 1e-6
      Small value to avoid division by 0.
  correct_bias : bool, default True
     Can be set to False to avoid correcting bias in Adam (e.g. like in Bert TF repository).
     Default True.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
code: def __init__(
      self,
      learning_rate=0.001,
      beta1=0.9,
      beta2=0.999,
      epsilon=1e-6,
      correct_bias=True,
      use_fused_step=True,
      **kwargs,
  ):
      super().__init__(
          use_fused_step=use_fused_step,
          learning_rate=learning_rate,
          **kwargs,
      )
      self.beta1 = beta1
      self.beta2 = beta2
      self.epsilon = epsilon
      self.correct_bias = correct_bias
      self.aggregate_num = max(
          1,
          min(
              50,
              int(os.getenv("MXNET_OPTIMIZER_AGGREGATION_SIZE", "4")),
          ),
      )
  
file_path: mxnet/python/mxnet/optimizer/adabelief.py
imports: import math
  import os
  import numpy
  from .optimizer import Optimizer, register
  from ..ndarray import zeros, clip, sqrt, square, full, NDArray
  from ..ndarray.contrib import mp_adabelief_update, adabelief_update, multi_mp_adabelief_update, multi_adabelief_update
label_desc: Use image node_id 1 to create a new AdaBelief object from inherited base classes: Optimizer with example: obj = AdaBelief(learning_rate, beta1, beta2, epsilon, correct_bias, use_fused_step)