class_docstr: Implements a pytorch variant of Adan
  Adan was proposed in
  Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models[J]. arXiv preprint arXiv:2208.06677, 2022.
  https://arxiv.org/abs/2208.06677
  Arguments:
      params (iterable): iterable of parameters to optimize or dicts defining parameter groups.
      lr (float, optional): learning rate. (default: 1e-3)
      betas (Tuple[float, float, flot], optional): coefficients used for computing
          running averages of gradient and its norm. (default: (0.98, 0.92, 0.99))
      eps (float, optional): term added to the denominator to improve
          numerical stability. (default: 1e-8)
      weight_decay (float, optional): decoupled weight decay (L2 penalty) (default: 0)
      no_prox (bool): how to perform the decoupled weight decay (default: False)
code: def restart_opt(self):
      for group in self.param_groups:
          group["step"] = 0
          for p in group["params"]:
              if p.requires_grad:
                  state = self.state[p]
                  # State initialization
  
                  # Exponential moving average of gradient values
                  state["exp_avg"] = torch.zeros_like(p)
                  # Exponential moving average of squared gradient values
                  state["exp_avg_sq"] = torch.zeros_like(p)
                  # Exponential moving average of gradient difference
                  state["exp_avg_diff"] = torch.zeros_like(p)
  
file_path: pytorch-image-models/timm/optim/adan.py
imports: import math
  import torch
  from torch.optim import Optimizer
label_desc: Use image node_id 2 for calling the Adan obj's underlying member method code with example usage: obj.restart_opt() without return types