class_docstr: LANS Optimizer.
  
  Referenced from 'Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes'
  (http://arxiv.org/abs/2006.13484)
  
  Parameters
  ----------
  learning_rate : float, default 0.001
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  beta1 : float, default 0.9
      Exponential decay rate for the first moment estimates.
  beta2 : float, default 0.999
      Exponential decay rate for the second moment estimates.
  epsilon : float, default 1e-6
      Small value to avoid division by 0.
  lower_bound : float, default None
      Lower limit of norm of weight
  upper_bound : float, default None
      Upper limit of norm of weight
  aggregate_num : int, default 4
      Number of weights to be aggregated in a list.
      They are passed to the optimizer for a single optimization step.
      In default, all the weights are aggregated.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
code: def __init__(
      self,
      learning_rate=0.001,
      beta1=0.9,
      beta2=0.999,
      epsilon=1e-6,
      lower_bound=None,
      upper_bound=None,
      aggregate_num=4,
      use_fused_step=True,
      **kwargs,
  ):
      assert aggregate_num <= 45, (
          "When use_fused_step is True, LAMB only supports aggregate_num <= 45,"
          " and receives {}".format(aggregate_num)
      )
      super(LANS, self).__init__(
          learning_rate=learning_rate,
          aggregate_num=aggregate_num,
          use_fused_step=use_fused_step,
          **kwargs,
      )
      self.beta1 = beta1
      self.beta2 = beta2
      self.epsilon = epsilon
      self.lower_bound = lower_bound
      self.upper_bound = upper_bound
  
file_path: mxnet/python/mxnet/optimizer/lans.py
imports: from __future__ import absolute_import
  import numpy
  from ..ndarray import zeros, clip, sqrt, where, square, ones_like, maximum, minimum
  from ..ndarray.contrib import multi_lans_update, multi_mp_lans_update
  from .optimizer import Optimizer, register
label_desc: Use image node_id 1 to create a new LANS object from inherited base classes: Optimizer with example: obj = LANS(learning_rate, beta1, beta2, epsilon, lower_bound, upper_bound, aggregate_num, use_fused_step)