class_docstr: Implements AdaBelief Optimizer Algorithm.
  It has been proposed in `AdaBelief Optimizer, adapting stepsizes by
  the belief in observed gradients`__.
  
  Arguments:
      params: iterable of parameters to optimize or dicts defining
          parameter groups
      lr: learning rate (default: 1e-2)
      betas: coefficients used for computing
          running averages of gradient and its square (default: (0.9, 0.999))
      eps: term added to the denominator to improve
          numerical stability (default: 0.001)
      weight_decay: weight decay (L2 penalty) (default: 0)
      amsgrad: whether to use the AMSGrad variant of this
          algorithm from the paper `On the Convergence of Adam and Beyond`_
          (default: False)
      weight_decouple:  If set as True, then the optimizer uses decoupled
          weight decay as in AdamW (default: False)
      fixed_decay : This is used when
          weight_decouple is set as True.
          When fixed_decay == True, the weight decay is performed as
          $W_{new} = W_{old} - W_{old} \times decay$.
          When fixed_decay == False, the weight decay is performed as
          $W_{new} = W_{old} - W_{old} \times decay \times lr$. Note that in
          this case, the weight decay ratio decreases with learning
          rate (lr).  (default: False)
      rectify: (default: False) If set as True, then perform the rectified
          update similar to RAdam
  
  Example:
      >>> import torch_optimizer as optim
      >>> optimizer = optim.AdaBelief(model.parameters(), lr=0.01)
      >>> optimizer.zero_grad()
      >>> loss_fn(model(input), target).backward()
      >>> optimizer.step()
  
  __ https://arxiv.org/abs/2010.07468
  
  Note:
      Reference code: https://github.com/juntang-zhuang/Adabelief-Optimizer
code: def __setstate__(self, state):
      super(AdaBelief, self).__setstate__(state)
      for group in self.param_groups:
          group.setdefault("amsgrad", False)
  
file_path: pytorch-optimizer/torch_optimizer/adabelief.py
imports: import math
  import torch
  from torch.optim.optimizer import Optimizer
  from .types import Betas2, OptFloat, OptLossClosure, Params
label_desc: Use image node_id 2 for calling the AdaBelief obj's underlying member method code with example usage: obj.__setstate__(state) without return types