class_docstr: Implements Adam algorithm.
  
  This implementation is modified from torch.optim.Adam based on:
  `Fixed Weight Decay Regularization in Adam`
  (see https://arxiv.org/abs/1711.05101)
  
  It has been proposed in `Adam: A Method for Stochastic Optimization`_.
  
  Args:
      params (iterable): iterable of parameters to optimize or dicts defining
          parameter groups
      lr (float, optional): learning rate (default: 1e-3)
      betas (Tuple[float, float], optional): coefficients used for computing
          running averages of gradient and its square (default: (0.9, 0.999))
      eps (float, optional): term added to the denominator to improve
          numerical stability (default: 1e-8)
      weight_decay (float, optional): weight decay (L2 penalty) (default: 0)
      amsgrad (boolean, optional): whether to use the AMSGrad variant of this
          algorithm from the paper `On the Convergence of Adam and Beyond`_
  
  .. _Adam\: A Method for Stochastic Optimization:
      https://arxiv.org/abs/1412.6980
  .. _On the Convergence of Adam and Beyond:
      https://openreview.net/forum?id=ryQu7f-RZ
class_docstr_tok: ["Implements","Adam","algorithm",".","This","implementation","is","modified","from","torch.optim.Adam","based","on",":","`","Fixed","Weight","Decay","Regularization","in","Adam","`","(","see","https",":","\/\/arxiv.org\/abs\/1711.05101",")","It","has","been","proposed","in","`","Adam",":","A","Method","for","Stochastic","Optimization","`","_",".","Args",":","params","(","iterable",")",":","iterable","of","parameters","to","optimize","or","dicts","defining","parameter","groups","lr","(","float",",","optional",")",":","learning","rate","(","default",":","1e-3",")","betas","(","Tuple","[","float",",","float","]",",","optional",")",":","coefficients","used","for","computing","running","averages","of","gradient","and","its","square","(","default",":","(","0.9",",","0.999",")",")","eps","(","float",",","optional",")",":","term","added","to","the","denominator","to","improve","numerical","stability","(","default",":","1e-8",")","weight_decay","(","float",",","optional",")",":","weight","decay","(","L2","penalty",")","(","default",":","0",")","amsgrad","(","boolean",",","optional",")",":","whether","to","use","the","AMSGrad","variant","of","this","algorithm","from","the","paper","`","On","the","Convergence","of","Adam","and","Beyond","`","_","..","_Adam\\",":","A","Method","for","Stochastic","Optimization",":","https",":","\/\/arxiv.org\/abs\/1412.6980","..","_On","the","Convergence","of","Adam","and","Beyond",":","https",":","\/\/openreview.net\/forum","?","id=ryQu7f-RZ"]
code: def supports_memory_efficient_fp16(self):
      return True
  
imports: import logging
  import math
  from dataclasses import dataclass, field
  from typing import List
  import torch
  import torch.distributed
  import torch.optim
  from fairseq.dataclass import FairseqDataclass
  from fairseq.optim import FairseqOptimizer, register_optimizer
  from omegaconf import II
  from lightseq.training.ops.pytorch.adam import LSAdam
label_desc: Use image node_id 2 for calling the Adam obj's underlying member method code with example usage: obj.supports_memory_efficient_fp16() and returns: True