class_docstr: A variant of the Adam optimizer that can handles both sparse and non-sparse updates.
  
  The original Adam algorithm maintains two moving-average accumulators for
  each trainable variable; the accumulators are updated at every step.
  This class provides lazier handling of gradient updates for sparse
  variables.  It only updates moving-average accumulators for sparse variable
  indices that appear in the current batch, rather than updating the
  accumulators for all indices. Compared with the original Adam optimizer,
  it can provide large improvements in model training throughput for some
  applications. However, it provides slightly different semantics than the
  original Adam algorithm, and may lead to different empirical results.
code: def _dense_step(self, group, param, grad):
      state = self.state[param]
  
      # State initialization
      if len(state) == 0:
          state["step"] = 0
          # Exponential moving average of gradient values
          state["exp_avg"] = torch.zeros_like(param.data)
          # Exponential moving average of squared gradient values
          state["exp_avg_sq"] = torch.zeros_like(param.data)
  
      exp_avg, exp_avg_sq = state["exp_avg"], state["exp_avg_sq"]
      beta1, beta2 = group["betas"]
  
      state["step"] += 1
      bias_correction1 = 1 - beta1 ** state["step"]
      bias_correction2 = 1 - beta2 ** state["step"]
  
      # Decay the first and second moment running average coefficient
      exp_avg.mul_(beta1).add_(1 - beta1, grad)
      exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad)
  
      denom = (exp_avg_sq.sqrt() / math.sqrt(bias_correction2)).add_(
          group["eps"]
      )
  
      step_size = group["lr"] / bias_correction1
  
      param.data.addcdiv_(-step_size, exp_avg, denom)
  
file_path: BigDL/python/nano/src/bigdl/nano/pytorch/optim/sparseadam.py
imports: import math
  import torch
  from torch.optim.optimizer import Optimizer
  from bigdl.nano.utils.common import invalidInputError
label_desc: Use image node_id 4 for calling the SparseAdam obj's underlying member method code with example usage: obj._dense_step(group, param, grad) without return types