class_docstr: NOTE: This class has been copied verbatim from the separate Dense and
  Sparse versions of Adam in Pytorch.
  
  Implements Adam algorithm with dense & sparse gradients.
  It has been proposed in Adam: A Method for Stochastic Optimization.
  
  Parameters
  ----------
  params : ``iterable``
      iterable of parameters to optimize or dicts defining parameter groups
  lr : ``float``, optional (default: 1e-3)
      The learning rate.
  betas : ``Tuple[float, float]``, optional (default: (0.9, 0.999))
      coefficients used for computing running averages of gradient
      and its square.
  eps : ``float``, optional, (default: 1e-8)
      A term added to the denominator to improve numerical stability.
code: def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8):
      if not 0.0 <= lr:
          raise ValueError("Invalid learning rate: {}".format(lr))
      if not 0.0 <= eps:
          raise ValueError("Invalid epsilon value: {}".format(eps))
      if not 0.0 <= betas[0] < 1.0:
          raise ValueError(
              "Invalid beta parameter at index 0: {}".format(betas[0])
          )
      if not 0.0 <= betas[1] < 1.0:
          raise ValueError(
              "Invalid beta parameter at index 1: {}".format(betas[1])
          )
      defaults = dict(lr=lr, betas=betas, eps=eps)
      super(DenseSparseAdam, self).__init__(params, defaults)
  
file_path: magnitude/pymagnitude/third_party/allennlp/training/optimizers.py
imports: from __future__ import division
  from __future__ import absolute_import
  import logging
  import re
  import math
  import torch
  from allennlp.common import Params, Registrable
label_desc: Use image node_id 1 to create a new DenseSparseAdam object from inherited base classes: torch.optim with example: obj = DenseSparseAdam(params, lr, betas, eps)