class_docstr: Gradient-descent optimizer with adaptive learning rate, first and second moment.
  
  Adaptive Moment Estimation uses a step-dependent learning rate,
  a first moment :math:`a` and a second moment :math:`b`, reminiscent of
  the momentum and velocity of a particle:
  
  .. math::
      x^{(t+1)} = x^{(t)} - \eta^{(t+1)} \frac{a^{(t+1)}}{\sqrt{b^{(t+1)}} + \epsilon },
  
  where the update rules for the two moments are given by
  
  .. math::
      a^{(t+1)} &= \beta_1 a^{(t)} + (1-\beta_1) \nabla f(x^{(t)}),\\
      b^{(t+1)} &= \beta_2 b^{(t)} + (1-\beta_2) (\nabla f(x^{(t)}))^{\odot 2},\\
      \eta^{(t+1)} &= \eta \frac{\sqrt{(1-\beta_2^{t+1})}}{(1-\beta_1^{t+1})}.
  
  Above, :math:`( \nabla f(x^{(t-1)}))^{\odot 2}` denotes the element-wise square operation,
  which means that each element in the gradient is multiplied by itself. The hyperparameters
  :math:`\beta_1` and :math:`\beta_2` can also be step-dependent. Initially, the first and
  second moment are zero.
  
  The shift :math:`\epsilon` avoids division by zero.
  
  For more details, see `arXiv:1412.6980 <https://arxiv.org/abs/1412.6980>`_.
  
  Args:
      stepsize (float): the user-defined hyperparameter :math:`\eta`
      beta1 (float): hyperparameter governing the update of the first and second moment
      beta2 (float): hyperparameter governing the update of the first and second moment
      eps (float): offset :math:`\epsilon` added for numerical stability
  
  .. note::
  
      When using ``torch``, ``tensorflow`` or ``jax`` interfaces, refer to :doc:`Gradients and training </introduction/interfaces>` for suitable optimizers.
docstr: Update the variables args to take a single optimization step. Flattens and unflattens
  the inputs to maintain nested iterables as the parameters of the optimization.
  
  Args:
      grad (tuple[ndarray]): the gradient of the objective
          function at point :math:`x^{(t)}`: :math:`\nabla f(x^{(t)})`
      args (tuple): the current value of the variables :math:`x^{(t)}`
  
  Returns:
      list: the new values :math:`x^{(t+1)}`
docstr_tok: ["Update","the","variables","args","to","take","a","single","optimization","step",".","Flattens","and","unflattens","the","inputs","to","maintain","nested","iterables","as","the","parameters","of","the","optimization",".","Args",":","grad","(","tuple","[","ndarray","]",")",":","the","gradient","of","the","objective","function","at","point",":","math",":","`","x^","{","(","t",")","}","`",":",":","math",":","`","\\nabla","f","(","x^","{","(","t",")","}",")","`","args","(","tuple",")",":","the","current","value","of","the","variables",":","math",":","`","x^","{","(","t",")","}","`","Returns",":","list",":","the","new","values",":","math",":","`","x^","{","(","t+1",")","}","`"]
code: def apply_grad(self, grad, args):
      r"""Update the variables args to take a single optimization step. Flattens and unflattens
      the inputs to maintain nested iterables as the parameters of the optimization.
  
      Args:
          grad (tuple[ndarray]): the gradient of the objective
              function at point :math:`x^{(t)}`: :math:`\nabla f(x^{(t)})`
          args (tuple): the current value of the variables :math:`x^{(t)}`
  
      Returns:
          list: the new values :math:`x^{(t+1)}`
      """
      args_new = list(args)
  
      if self.accumulation is None:
          self.accumulation = {
              "fm": [0] * len(args),
              "sm": [0] * len(args),
              "t": 0,
          }
  
      self.accumulation["t"] += 1
  
      # Update step size (instead of correcting for bias)
      new_stepsize = (
          self.stepsize
          * sqrt(1 - self.beta2 ** self.accumulation["t"])
          / (1 - self.beta1 ** self.accumulation["t"])
      )
  
      trained_index = 0
      for index, arg in enumerate(args):
          if getattr(arg, "requires_grad", False):
              self._update_accumulation(index, grad[trained_index])
              args_new[index] = arg - new_stepsize * self.accumulation[
                  "fm"
              ][index] / (
                  sqrt(self.accumulation["sm"][index]) + self.eps
              )
  
              trained_index += 1
  
      return args_new
  
file_path: pennylane/pennylane/optimize/adam.py
imports: from numpy import sqrt
  from .gradient_descent import GradientDescentOptimizer
label_desc: Use image node_id 2 for calling the AdamOptimizer obj's underlying member method code with example usage: obj.apply_grad(grad, args) and returns: args_new