class_docstr: The FTML optimizer.
  
  This class implements the optimizer described in
  *FTML - Follow the Moving Leader in Deep Learning*,
  available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
  
  Denote time step by t. The optimizer updates the weight by::
  
      rescaled_grad = clip(grad * rescale_grad, clip_gradient) + wd * weight
      v = beta2 * v + (1 - beta2) * square(rescaled_grad)
      d_t = (1 - power(beta1, t)) / lr * (square_root(v / (1 - power(beta2, t))) + epsilon)
      z = beta1 * z + (1 - beta1) * rescaled_grad - (d_t - beta1 * d_(t-1)) * weight
      weight = - z / d_t
  
  For details of the update algorithm, see :class:`~mxnet.ndarray.ftml_update`.
  
  This optimizer accepts the following parameters in addition to those accepted
  by :class:`.Optimizer`.
  
  Parameters
  ----------
  learning_rate : float, default 0.0025
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  beta1 : float, default 0.6
      0 < beta1 < 1. Generally close to 0.5.
  beta2 : float, default 0.999
      0 < beta2 < 1. Generally close to 1.
  epsilon : float, default 1e-8
      Small value to avoid division by 0.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
docstr: Perform a fused optimization step using gradients and states.
  Fused kernel is used for update.
  
  Parameters
  ----------
  indices : list of int
      List of unique indices of the parameters into the individual learning rates
      and weight decays. Learning rates and weight decay may be set via `set_lr_mult()`
      and `set_wd_mult()`, respectively.
  weights : list of NDArray
      List of parameters to be updated.
  grads : list of NDArray
      List of gradients of the objective with respect to this parameter.
  states : List of any obj
      List of state returned by `create_state()`.
docstr_tok: ["Perform","a","fused","optimization","step","using","gradients","and","states",".","Fused","kernel","is","used","for","update",".","Parameters","--","--","--","--","--","indices",":","list","of","int","List","of","unique","indices","of","the","parameters","into","the","individual","learning","rates","and","weight","decays",".","Learning","rates","and","weight","decay","may","be","set","via","`","set_lr_mult","(",")","`","and","`","set_wd_mult","(",")","`",",","respectively",".","weights",":","list","of","NDArray","List","of","parameters","to","be","updated",".","grads",":","list","of","NDArray","List","of","gradients","of","the","objective","with","respect","to","this","parameter",".","states",":","List","of","any","obj","List","of","state","returned","by","`","create_state","(",")","`","."]
code: def fused_step(self, indices, weights, grads, states):
      """Perform a fused optimization step using gradients and states.
      Fused kernel is used for update.
  
      Parameters
      ----------
      indices : list of int
          List of unique indices of the parameters into the individual learning rates
          and weight decays. Learning rates and weight decay may be set via `set_lr_mult()`
          and `set_wd_mult()`, respectively.
      weights : list of NDArray
          List of parameters to be updated.
      grads : list of NDArray
          List of gradients of the objective with respect to this parameter.
      states : List of any obj
          List of state returned by `create_state()`.
      """
      for index, weight, grad, state in zip(
          indices, weights, grads, states
      ):
          self._update_count(index)
          lr = self._get_lr(index)
          wd = self._get_wd(index)
          t = self._index_update_count[index]
  
          kwargs = {
              "beta1": self.beta1,
              "beta2": self.beta2,
              "epsilon": self.epsilon,
              "rescale_grad": self.rescale_grad,
              "t": t,
          }
          if self.clip_gradient:
              kwargs["clip_grad"] = self.clip_gradient
  
          d, v, z = state
  
          # update weight with fused kernel
          ftml_update(
              weight, grad, d, v, z, out=weight, lr=lr, wd=wd, **kwargs
          )
  
file_path: mxnet/python/mxnet/optimizer/ftml.py
imports: from __future__ import absolute_import
  from ..ndarray import zeros, clip, sqrt, square
  from ..ndarray import ftml_update
  from .optimizer import Optimizer, register
label_desc: Use image node_id 4 for calling the FTML obj's underlying member method code with example usage: obj.fused_step(indices, weights, grads, states) without return types