class_docstr: A variant of the Adam optimizer that can handles both sparse and non-sparse updates.
  
  The original Adam algorithm maintains two moving-average accumulators for
  each trainable variable; the accumulators are updated at every step.
  This class provides lazier handling of gradient updates for sparse
  variables.  It only updates moving-average accumulators for sparse variable
  indices that appear in the current batch, rather than updating the
  accumulators for all indices. Compared with the original Adam optimizer,
  it can provide large improvements in model training throughput for some
  applications. However, it provides slightly different semantics than the
  original Adam algorithm, and may lead to different empirical results.
docstr: Construct a new SparseAdam optimizer.
  
  param lr: A `Tensor` or a floating point value. or a schedule
      that is a `tf.keras.optimizers.schedules.LearningRateSchedule`
      The learning rate.
  param beta_1: A `float` value or a constant `float` tensor.
      The exponential decay rate for the 1st moment estimates.
  param beta_2: A `float` value or a constant `float` tensor.
      The exponential decay rate for the 2nd moment estimates.
  param epsilon: A small constant for numerical stability.
      This epsilon is "epsilon hat" in
      [Adam: A Method for Stochastic Optimization. Kingma et al., 2014]
      (http://arxiv.org/abs/1412.6980) (in the formula just
      before Section 2.1), not the epsilon in Algorithm 1 of the paper.
code: def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8):
      """
      Construct a new SparseAdam optimizer.
  
      param lr: A `Tensor` or a floating point value. or a schedule
          that is a `tf.keras.optimizers.schedules.LearningRateSchedule`
          The learning rate.
      param beta_1: A `float` value or a constant `float` tensor.
          The exponential decay rate for the 1st moment estimates.
      param beta_2: A `float` value or a constant `float` tensor.
          The exponential decay rate for the 2nd moment estimates.
      param epsilon: A small constant for numerical stability.
          This epsilon is "epsilon hat" in
          [Adam: A Method for Stochastic Optimization. Kingma et al., 2014]
          (http://arxiv.org/abs/1412.6980) (in the formula just
          before Section 2.1), not the epsilon in Algorithm 1 of the paper.
      """
      if not 0.0 < lr:
          invalidInputError(
              False, "Invalid learning rate: {}".format(lr)
          )
      if not 0.0 < eps:
          invalidInputError(
              False, "Invalid epsilon value: {}".format(eps)
          )
      if not 0.0 <= betas[0] < 1.0:
          invalidInputError(
              False,
              "Invalid beta parameter at index 0: {}".format(betas[0]),
          )
      if not 0.0 <= betas[1] < 1.0:
          invalidInputError(
              False,
              "Invalid beta parameter at index 1: {}".format(betas[1]),
          )
  
      defaults = dict(lr=lr, betas=betas, eps=eps)
      super(SparseAdam, self).__init__(params, defaults)
  
file_path: BigDL/python/nano/src/bigdl/nano/pytorch/optim/sparseadam.py
imports: import math
  import torch
  from torch.optim.optimizer import Optimizer
  from bigdl.nano.utils.common import invalidInputError
label_desc: Use image node_id 1 to create a new SparseAdam object from inherited base classes: Optimizer with example: obj = SparseAdam(params, lr, betas, eps)