class_docstr: Mirror MADGRAD_: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic 
  Optimization.
  
  .. _MADGRAD: https://arxiv.org/abs/2101.11075
  
  Mirror MADGRAD uses the weighting and momentum of MADGRAD but uses mirror descent
  rather than dual averaging as the base method. In general, the mirror variant works 
  better than standard MADGRAD on problems where generalization gap is not an issue, 
  such as large Transformer model training. On CIFAR-10/Image-Net and smaller NLP models
  the standard variant should be prefered. The Mirror variant is more numerically stable
  which may help with large model training.
  
  Currently does not support sparse gradients.
  
  Arguments:
      params (iterable): 
          Iterable of parameters to optimize or dicts defining parameter groups.
      lr (float): 
          Learning rate (default: 1e-2).
      momentum (float): 
          Momentum value in the range [0,1) (default: 0.9).
      weight_decay (float): 
          Weight decay, i.e. a L2 penalty (default: 0).
      eps (float): 
          Term added to the denominator outside of the root operation to improve numerical stability. (default: 0).
          This parameter is less important in MADGRAD than in Adam. A value of 0 will likely give the best results.
      decouple_decay (bool):
          Apply AdamW style decoupled weight decay (EXPERIMENTAL). 
          Application of decay occurs before the step.
code: def supports_flat_params(self) -> bool:
      return True
  
file_path: madgrad/madgrad/mirror_madgrad.py
imports: import math
  import torch
  import torch.optim
  import math
  from typing import TYPE_CHECKING, Any, Callable, Optional
label_desc: Use image node_id 3 for calling the MirrorMADGRAD obj's underlying member method code with example usage: obj.supports_flat_params() and returns: True