class_docstr: Implements AdamP algorithm.
  
  It has been proposed in `Slowing Down the Weight Norm Increase in
  Momentum-based Optimizers`__
  
  Arguments:
      params: iterable of parameters to optimize or dicts defining
          parameter groups
      lr: learning rate (default: 1e-3)
      betas: coefficients used for computing
          running averages of gradient and its square (default: (0.9, 0.999))
      eps: term added to the denominator to improve
          numerical stability (default: 1e-8)
      weight_decay: weight decay (L2 penalty) (default: 0)
      delta: threhold that determines whether a set of parameters is scale
          invariant or not (default: 0.1)
      wd_ratio: relative weight decay applied on scale-invariant parameters
          compared to that applied on scale-variant parameters (default: 0.1)
      nesterov: enables Nesterov momentum (default: False)
  
  
  Example:
      >>> import torch_optimizer as optim
      >>> optimizer = optim.AdamP(model.parameters(), lr=0.1)
      >>> optimizer.zero_grad()
      >>> loss_fn(model(input), target).backward()
      >>> optimizer.step()
  
   __ https://arxiv.org/abs/2006.08217
  
  Note:
      Reference code: https://github.com/clovaai/AdamP
code: def _projection(self, p, grad, perturb, delta, wd_ratio, eps):
      wd = 1
      expand_size = [-1] + [1] * (len(p.shape) - 1)
      for view_func in [self._channel_view, self._layer_view]:
          cosine_sim = self._cosine_similarity(
              grad, p.data, eps, view_func
          )
  
          if cosine_sim.max() < delta / math.sqrt(
              view_func(p.data).size(1)
          ):
              p_n = p.data / view_func(p.data).norm(dim=1).view(
                  expand_size
              ).add_(eps)
              perturb -= p_n * view_func(p_n * perturb).sum(dim=1).view(
                  expand_size
              )
              wd = wd_ratio
  
              return perturb, wd
  
      return perturb, wd
  
file_path: pytorch-optimizer/torch_optimizer/adamp.py
imports: import math
  import torch
  from torch.optim.optimizer import Optimizer
  from .types import Betas2, OptFloat, OptLossClosure, Params
label_desc: Use image node_id 5 for calling the AdamP obj's underlying member method code with example usage: obj._projection(p, grad, perturb, delta, wd_ratio, eps) and returns: perturb,  wd, perturb,  wd