class_docstr: Gradient-descent optimizer with past-gradient-dependent
  learning rate in each dimension.
  
  Adagrad adjusts the learning rate for each parameter :math:`x_i`
  in :math:`x` based on past gradients. We therefore have to consider
  each parameter update individually,
  
  .. math::
      x^{(t+1)}_i = x^{(t)}_i - \eta_i^{(t+1)} \partial_{w_i} f(x^{(t)}),
  
  where the gradient is replaced by a (scalar) partial derivative.
  
  The learning rate in step :math:`t` is given by
  
  .. math::
      \eta_i^{(t+1)} = \frac{ \eta_{\mathrm{init}} }{ \sqrt{a_i^{(t+1)} + \epsilon } },
      ~~~ a_i^{(t+1)} = \sum_{k=1}^t (\partial_{x_i} f(x^{(k)}))^2.
  
  The offset :math:`\epsilon` avoids division by zero.
  
  :math:`\eta` is the step size, a user defined parameter.
  
  Args:
      stepsize (float): the user-defined hyperparameter :math:`\eta`
      eps (float): offset :math:`\epsilon` added for numerical stability
  
  .. note::
  
      When using ``torch``, ``tensorflow`` or ``jax`` interfaces, refer to :doc:`Gradients and training </introduction/interfaces>` for suitable optimizers.
docstr: Update the variables in args to take a single optimization step. Flattens and unflattens
  the inputs to maintain nested iterables as the parameters of the optimization.
  
  Args:
      grad (tuple[array]): the gradient of the objective
          function at point :math:`x^{(t)}`: :math:`\nabla f(x^{(t)})`
      args (tuple): the current value of the variables :math:`x^{(t)}`
  
  Returns:
      list: the new values :math:`x^{(t+1)}`
docstr_tok: ["Update","the","variables","in","args","to","take","a","single","optimization","step",".","Flattens","and","unflattens","the","inputs","to","maintain","nested","iterables","as","the","parameters","of","the","optimization",".","Args",":","grad","(","tuple","[","array","]",")",":","the","gradient","of","the","objective","function","at","point",":","math",":","`","x^","{","(","t",")","}","`",":",":","math",":","`","\\nabla","f","(","x^","{","(","t",")","}",")","`","args","(","tuple",")",":","the","current","value","of","the","variables",":","math",":","`","x^","{","(","t",")","}","`","Returns",":","list",":","the","new","values",":","math",":","`","x^","{","(","t+1",")","}","`"]
code: def apply_grad(self, grad, args):
      r"""Update the variables in args to take a single optimization step. Flattens and unflattens
      the inputs to maintain nested iterables as the parameters of the optimization.
  
      Args:
          grad (tuple[array]): the gradient of the objective
              function at point :math:`x^{(t)}`: :math:`\nabla f(x^{(t)})`
          args (tuple): the current value of the variables :math:`x^{(t)}`
  
      Returns:
          list: the new values :math:`x^{(t+1)}`
      """
      args_new = list(args)
  
      if self.accumulation is None:
          self.accumulation = [0.0] * len(args)
  
      trained_index = 0
      for index, arg in enumerate(args):
          if getattr(arg, "requires_grad", False):
              self._update_accumulation(index, grad[trained_index])
  
              coeff = self.stepsize / sqrt(
                  self.accumulation[index] + self.eps
              )
              args_new[index] = arg - coeff * grad[trained_index]
  
              trained_index += 1
  
      return args_new
  
imports: from numpy import sqrt
  from .gradient_descent import GradientDescentOptimizer
label_desc: Use image node_id 2 for calling the AdagradOptimizer obj's underlying member method code with example usage: obj.apply_grad(grad, args) and returns: args_new