class_docstr: The SGD optimizer with momentum and weight decay.
  
  If the storage types of grad is ``row_sparse`` and ``lazy_update`` is True,     **lazy updates** are applied by::
  
      for row in grad.indices:
          rescaled_grad[row] = clip(rescale_grad * grad[row] + wd * weight[row], clip_gradient)
          state[row] = momentum[row] * state[row] + lr * rescaled_grad[row]
          weight[row] = weight[row] - state[row]
  
  The sparse update only updates the momentum for the weights whose row_sparse
  gradient indices appear in the current batch, rather than updating it for all
  indices. Compared with the original update, it can provide large
  improvements in model training throughput for some applications. However, it
  provides slightly different semantics than the original update, and
  may lead to different empirical results.
  
  In the case when ``update_on_kvstore`` is set to False (either globally via
  MXNET_UPDATE_ON_KVSTORE=0 environment variable or as a parameter in
  :class:`~mxnet.gluon.Trainer`) SGD optimizer can perform aggregated update
  of parameters, which may lead to improved performance. The aggregation size
  is controlled by ``aggregate_num`` and defaults to 4.
  
  Otherwise, **standard updates** are applied by::
  
      rescaled_grad = clip(rescale_grad * grad, clip_gradient)) + wd * weight
      state = momentum * state + lr * rescaled_grad
      weight = weight - state
  
  For details of the update algorithm see
  :class:`~mxnet.ndarray.sgd_update` and :class:`~mxnet.ndarray.sgd_mom_update`.
  
  This optimizer accepts the following parameters in addition to those accepted
  by :class:`.Optimizer`.
  
  Parameters
  ----------
  learning_rate : float, default 0.1
      The initial learning rate. If None, the optimization will use the
      learning rate from ``lr_scheduler``. If not None, it will overwrite
      the learning rate in ``lr_scheduler``. If None and ``lr_scheduler``
      is also None, then it will be set to 0.01 by default.
  momentum : float, default 0.
      The momentum value.
  lazy_update : bool, default False
      Default is False. If True, lazy updates are applied         if the storage types of weight and grad are both ``row_sparse``.
  multi_precision: bool, default False
      Flag to control the internal precision of the optimizer.
      False: results in using the same precision as the weights (default),
      True: makes internal 32-bit copy of the weights and applies gradients
      in 32-bit precision even if actual weights used in the model have lower precision.
      Turning this on can improve convergence and accuracy when training with float16.
  aggregate_num : int, default 1
      Number of weights to be aggregated in a list.
      They are passed to the optimizer for a single optimization step.
  use_fused_step : bool, default True
      Whether or not to use fused kernels for optimizer.
      When use_fused_step=False, step is called,
      otherwise, fused_step is called.
class_docstr_tok: ["The","SGD","optimizer","with","momentum","and","weight","decay",".","If","the","storage","types","of","grad","is","``","row_sparse","``","and","``","lazy_update","``","is","True",",","*","*","lazy","updates","*","*","are","applied","by",":",":","for","row","in","grad.indices",":","rescaled_grad","[","row","]","=","clip","(","rescale_grad","*","grad","[","row","]","+","wd","*","weight","[","row","]",",","clip_gradient",")","state","[","row","]","=","momentum","[","row","]","*","state","[","row","]","+","lr","*","rescaled_grad","[","row","]","weight","[","row","]","=","weight","[","row","]","-","state","[","row","]","The","sparse","update","only","updates","the","momentum","for","the","weights","whose","row_sparse","gradient","indices","appear","in","the","current","batch",",","rather","than","updating","it","for","all","indices",".","Compared","with","the","original","update",",","it","can","provide","large","improvements","in","model","training","throughput","for","some","applications",".","However",",","it","provides","slightly","different","semantics","than","the","original","update",",","and","may","lead","to","different","empirical","results",".","In","the","case","when","``","update_on_kvstore","``","is","set","to","False","(","either","globally","via","MXNET_UPDATE_ON_KVSTORE=0","environment","variable","or","as","a","parameter","in",":","class",":","`","~mxnet.gluon.Trainer","`",")","SGD","optimizer","can","perform","aggregated","update","of","parameters",",","which","may","lead","to","improved","performance",".","The","aggregation","size","is","controlled","by","``","aggregate_num","``","and","defaults","to","4",".","Otherwise",",","*","*","standard","updates","*","*","are","applied","by",":",":","rescaled_grad","=","clip","(","rescale_grad","*","grad",",","clip_gradient",")",")","+","wd","*","weight","state","=","momentum","*","state","+","lr","*","rescaled_grad","weight","=","weight","-","state","For","details","of","the","update","algorithm","see",":","class",":","`","~mxnet.ndarray.sgd_update","`","and",":","class",":","`","~mxnet.ndarray.sgd_mom_update","`",".","This","optimizer","accepts","the","following","parameters","in","addition","to","those","accepted","by",":","class",":","`",".Optimizer","`",".","Parameters","--","--","--","--","--","learning_rate",":","float",",","default","0.1","The","initial","learning","rate",".","If","None",",","the","optimization","will","use","the","learning","rate","from","``","lr_scheduler","``",".","If","not","None",",","it","will","overwrite","the","learning","rate","in","``","lr_scheduler","``",".","If","None","and","``","lr_scheduler","``","is","also","None",",","then","it","will","be","set","to","0.01","by","default",".","momentum",":","float",",","default","0",".","The","momentum","value",".","lazy_update",":","bool",",","default","False","Default","is","False",".","If","True",",","lazy","updates","are","applied","if","the","storage","types","of","weight","and","grad","are","both","``","row_sparse","``",".","multi_precision",":","bool",",","default","False","Flag","to","control","the","internal","precision","of","the","optimizer",".","False",":","results","in","using","the","same","precision","as","the","weights","(","default",")",",","True",":","makes","internal","32-bit","copy","of","the","weights","and","applies","gradients","in","32-bit","precision","even","if","actual","weights","used","in","the","model","have","lower","precision",".","Turning","this","on","can","improve","convergence","and","accuracy","when","training","with","float16",".","aggregate_num",":","int",",","default","1","Number","of","weights","to","be","aggregated","in","a","list",".","They","are","passed","to","the","optimizer","for","a","single","optimization","step",".","use_fused_step",":","bool",",","default","True","Whether","or","not","to","use","fused","kernels","for","optimizer",".","When","use_fused_step=False",",","step","is","called",",","otherwise",",","fused_step","is","called","."]
code: def __init__(
      self,
      learning_rate=0.1,
      momentum=0.0,
      lazy_update=False,
      multi_precision=False,
      use_fused_step=True,
      aggregate_num=1,
      **kwargs,
  ):
      super(SGD, self).__init__(
          learning_rate=learning_rate,
          multi_precision=multi_precision,
          aggregate_num=aggregate_num,
          use_fused_step=use_fused_step,
          **kwargs,
      )
      if not self.use_fused_step:
          assert (
              not lazy_update
          ), "When use_fused_step is set to False, lazy_update has to be turned off."
      if lazy_update:
          assert (
              not multi_precision
          ), "When lazy_update is set to True, multi_precision has be turned off."
      self.momentum = momentum
      self.lazy_update = lazy_update
  
file_path: mxnet/python/mxnet/optimizer/sgd.py
imports: from __future__ import absolute_import
  import numpy
  from ..ndarray import zeros, clip
  from ..ndarray import sgd_update, sgd_mom_update, mp_sgd_update, mp_sgd_mom_update, multi_sgd_update, multi_sgd_mom_update, multi_mp_sgd_update, multi_mp_sgd_mom_update
  from .optimizer import Optimizer, register
  from .utils import _flatten_list
label_desc: Use image node_id 1 to create a new SGD object from inherited base classes: Optimizer with example: obj = SGD(learning_rate, momentum, lazy_update, multi_precision, use_fused_step, aggregate_num)