class_docstr: Applies an `Optimizer` on a set of Parameters. Trainer should
  be used together with `autograd`.
  
  .. note::
  
      For the following cases, updates will always happen on kvstore,
      i.e., you cannot set update_on_kvstore=False.
  
      - dist kvstore with sparse weights or sparse gradients
      - dist async kvstore
      - `optimizer.lr_scheduler` is not None
  
  Parameters
  ----------
  params : Dict
      The set of parameters to optimize.
  optimizer : str or Optimizer
      The optimizer to use. See
      `help <https://mxnet.apache.org/api/python/docs/api/optimizer/index.html>`_
      on Optimizer for a list of available optimizers.
  optimizer_params : dict
      Key-word arguments to be passed to optimizer constructor. For example,
      `{'learning_rate': 0.1}`. All optimizers accept learning_rate, wd (weight decay),
      clip_gradient, and lr_scheduler. See each optimizer's
      constructor for a list of additional supported arguments.
  kvstore : str or KVStore
      kvstore type for multi-gpu and distributed training. See help on
      :func:`mxnet.kvstore.create` for more information.
  compression_params : dict
      Specifies type of gradient compression and additional arguments depending
      on the type of compression being used. For example, 2bit compression requires a threshold.
      Arguments would then be {'type':'2bit', 'threshold':0.5}
      See mxnet.KVStore.set_gradient_compression method for more details on gradient compression.
  update_on_kvstore : bool, default None
      Whether to perform parameter updates on kvstore. If None and optimizer.aggregate_num <= 1,
      then trainer will choose the more suitable option depending on the type of kvstore.
      If None and optimizer.aggregate_num > 1, `update_on_kvstore` is set to False.
      If the `update_on_kvstore` argument is provided,
      environment variable `MXNET_UPDATE_ON_KVSTORE` will be ignored.
  
  Properties
  ----------
  learning_rate : float
      The current learning rate of the optimizer. Given an Optimizer object
      optimizer, its learning rate can be accessed as optimizer.learning_rate.
class_docstr_tok: ["Applies","an","`","Optimizer","`","on","a","set","of","Parameters",".","Trainer","should","be","used","together","with","`","autograd","`",".","..","note",":",":","For","the","following","cases",",","updates","will","always","happen","on","kvstore",",","i.e.",",","you","can","not","set","update_on_kvstore=False",".","-","dist","kvstore","with","sparse","weights","or","sparse","gradients","-","dist","async","kvstore","-","`","optimizer.lr_scheduler","`","is","not","None","Parameters","--","--","--","--","--","params",":","Dict","The","set","of","parameters","to","optimize",".","optimizer",":","str","or","Optimizer","The","optimizer","to","use",".","See","`","help","<","https",":","\/\/mxnet.apache.org\/api\/python\/docs\/api\/optimizer\/index.html",">","`","_","on","Optimizer","for","a","list","of","available","optimizers",".","optimizer_params",":","dict","Key-word","arguments","to","be","passed","to","optimizer","constructor",".","For","example",",","`","{","'learning_rate","'",":","0.1","}","`",".","All","optimizers","accept","learning_rate",",","wd","(","weight","decay",")",",","clip_gradient",",","and","lr_scheduler",".","See","each","optimizer's","constructor","for","a","list","of","additional","supported","arguments",".","kvstore",":","str","or","KVStore","kvstore","type","for","multi-gpu","and","distributed","training",".","See","help","on",":","func",":","`","mxnet.kvstore.create","`","for","more","information",".","compression_params",":","dict","Specifies","type","of","gradient","compression","and","additional","arguments","depending","on","the","type","of","compression","being","used",".","For","example",",","2bit","compression","requires","a","threshold",".","Arguments","would","then","be","{","'type","'",":","'2bit","'",",","'threshold':0.5","}","See","mxnet.KVStore.set_gradient_compression","method","for","more","details","on","gradient","compression",".","update_on_kvstore",":","bool",",","default","None","Whether","to","perform","parameter","updates","on","kvstore",".","If","None","and","optimizer.aggregate_num","<","=","1",",","then","trainer","will","choose","the","more","suitable","option","depending","on","the","type","of","kvstore",".","If","None","and","optimizer.aggregate_num",">","1",",","`","update_on_kvstore","`","is","set","to","False",".","If","the","`","update_on_kvstore","`","argument","is","provided",",","environment","variable","`","MXNET_UPDATE_ON_KVSTORE","`","will","be","ignored",".","Properties","--","--","--","--","--","learning_rate",":","float","The","current","learning","rate","of","the","optimizer",".","Given","an","Optimizer","object","optimizer",",","its","learning","rate","can","be","accessed","as","optimizer.learning_rate","."]
docstr: For each parameter, reduce the gradients from different devices.
  
  Should be called after `autograd.backward()`, outside of `record()` scope,
  and before `trainer.update()`.
  
  For normal parameter updates, `step()` should be used, which internally calls
  `allreduce_grads()` and then `update()`. However, if you need to get the reduced
  gradients to perform certain transformation, such as in gradient clipping, then
  you may want to manually call `allreduce_grads()` and `update()` separately.
docstr_tok: ["For","each","parameter",",","reduce","the","gradients","from","different","devices",".","Should","be","called","after","`","autograd.backward","(",")","`",",","outside","of","`","record","(",")","`","scope",",","and","before","`","trainer.update","(",")","`",".","For","normal","parameter","updates",",","`","step","(",")","`","should","be","used",",","which","internally","calls","`","allreduce_grads","(",")","`","and","then","`","update","(",")","`",".","However",",","if","you","need","to","get","the","reduced","gradients","to","perform","certain","transformation",",","such","as","in","gradient","clipping",",","then","you","may","want","to","manually","call","`","allreduce_grads","(",")","`","and","`","update","(",")","`","separately","."]
code: def allreduce_grads(self):
      """For each parameter, reduce the gradients from different devices.
  
      Should be called after `autograd.backward()`, outside of `record()` scope,
      and before `trainer.update()`.
  
      For normal parameter updates, `step()` should be used, which internally calls
      `allreduce_grads()` and then `update()`. However, if you need to get the reduced
      gradients to perform certain transformation, such as in gradient clipping, then
      you may want to manually call `allreduce_grads()` and `update()` separately.
      """
      if not self._kv_initialized:
          self._init_kvstore()
      if self._params_to_init:
          self._init_params()
      assert not (self._kvstore and self._update_on_kvstore), (
          "allreduce_grads() when parameters are updated on kvstore "
          "is not supported. Try setting `update_on_kvstore` "
          "to False when creating trainer."
      )
  
      self._allreduce_grads()
  
imports: import warnings
  from collections import OrderedDict
  from ..None import optimizer
  from ..model import _create_kvstore, _create_sparse_kvstore
  from .parameter import Parameter
  from ..kvstore import KVStore
label_desc: Use image node_id 14 for calling the Trainer obj's underlying member method code with example usage: obj.allreduce_grads() without return types