class_docstr: This AdamW implementation is copied from Huggingface.
  We modified it with Adagrad scaling by rms of a weight tensor
  Implements Adam algorithm with weight decay fix as introduced in [Decoupled Weight Decay
  Regularization](https://arxiv.org/abs/1711.05101).
  Parameters:
      params (`Iterable[nn.parameter.Parameter]`):
          Iterable of parameters to optimize or dictionaries defining parameter groups.
      lr (`float`, *optional*, defaults to 1e-3):
          The learning rate to use.
      betas (`Tuple[float,float]`, *optional*, defaults to (0.9, 0.999)):
          Adam's betas parameters (b1, b2).
      eps (`float`, *optional*, defaults to 1e-6):
          Adam's epsilon for numerical stability.
      weight_decay (`float`, *optional*, defaults to 0):
          Decoupled weight decay to apply.
      correct_bias (`bool`, *optional*, defaults to `True`):
          Whether or not to correct bias in Adam (for instance, in Bert TF repository they use `False`).
      no_deprecation_warning (`bool`, *optional*, defaults to `False`):
          A flag used to disable the deprecation warning (set to `True` to disable the warning).
code: def _rms(tensor):
      return tensor.norm(2) / (tensor.numel() ** 0.5)
  
file_path: bert-ultrafast/training/cramming/backend/optimizers/nanoT5_optimizer.py
imports: import torch
  import math
  from typing import Iterable, Tuple
  from torch import nn
label_desc: Use image node_id 2 for calling the AdamWScale obj's underlying member method code with example usage: obj._rms(tensor) and returns: unknown