class_docstr: The FTML optimizer.
  
  This class implements the optimizer described in
  *FTML - Follow the Moving Leader in Deep Learning*,
  available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
  
  Denote time step by t. The optimizer updates the weight by::
  
      rescaled_grad = clip(grad * rescale_grad, clip_gradient) + wd * weight
      v = beta2 * v + (1 - beta2) * square(rescaled_grad)
      d_t = (1 - power(beta1, t)) / lr * (square_root(v / (1 - power(beta2, t))) + epsilon)
      z = beta1 * z + (1 - beta1) * rescaled_grad - (d_t - beta1 * d_(t-1)) * weight
      weight = - z / d_t
  
  For details of the update algorithm, see :class:`~mxnet.ndarray.ftml_update`.
  
  This optimizer accepts the following parameters in addition to those accepted
  by :class:`.Optimizer`.
  
  Parameters
  ----------
  learning_rate : float, default 0.0025
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  beta1 : float, default 0.6
      0 < beta1 < 1. Generally close to 0.5.
  beta2 : float, default 0.999
      0 < beta2 < 1. Generally close to 1.
  epsilon : float, default 1e-8
      Small value to avoid division by 0.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
code: def create_state(self, index, weight):
      return (
          zeros(
              weight.shape, weight.context, dtype=weight.dtype
          ),  # d_0
          zeros(
              weight.shape, weight.context, dtype=weight.dtype
          ),  # v_0
          zeros(weight.shape, weight.context, dtype=weight.dtype),
      )
  
file_path: mxnet/python/mxnet/optimizer/ftml.py
imports: from __future__ import absolute_import
  from ..ndarray import zeros, clip, sqrt, square
  from ..ndarray import ftml_update
  from .optimizer import Optimizer, register
label_desc: Use image node_id 2 for calling the FTML obj's underlying member method code with example usage: obj.create_state(index, weight) and returns: zeros,  zeros,  zeros