class_docstr: Implementation of a parallel GPU Updater.
  
  This is an implementation of :class:`Updater` that uses multiple GPUs.
  It behaves similarly to
  :class:`~chainer.training.updaters.StandardUpdater`.
  The update routine is modified to support data-parallel computation
  on multiple GPUs in one machine.
  It is based on synchronous parallel SGD: it
  parallelizes the gradient computation over a mini-batch, and updates the
  parameters only in the main device.
  
  Args:
      iterator: Dataset iterator for the training dataset. It can also be a
          dictionary that maps strings to iterators.
          If this is just an iterator, then the
          iterator is registered by the name ``'main'``.
      optimizer: Optimizer to update parameters. It can also be a dictionary
          that maps strings to optimizers.
          If this is just an optimizer, then the optimizer is
          registered by the name ``'main'``.
      converter: Converter function to build input arrays. Each batch
          extracted by the main iterator is split equally between the
          devices and then passed with corresponding ``device`` option to
          this function. :func:`~chainer.dataset.concat_examples` is used by
          default.
      models: Dictionary of models. The main model should be the same model
          attached to the ``'main'`` optimizer.
      devices: Dictionary of devices to which the training data is sent. The
          devices should be arranged in a dictionary with the same structure
          as ``models``.
      loss_func: Loss function. The model is used as a loss function by
          default.
      loss_scale (float): Loss scaling factor. Loss scaling is a useful
          technique to mitigate vanishing gradient issue that tends to happen
          when low precision data type like float16 is used during training.
          If you set loss scaling factor, gradients of loss values are to be
          multiplied by the factor before backprop starts. The factor is
          propagated to whole gradients in a computational graph along the
          backprop. The gradients of parameters are divided by the factor
          just before the parameters are to be updated.
      auto_new_epoch (bool):  If ``True``,
          :meth:`~chainer.Optimizer.new_epoch` of the main optimizer is
          automatically called when the ``is_new_epoch`` attribute of the
          main iterator is ``True``.
class_docstr_tok: ["Implementation","of","a","parallel","GPU","Updater",".","This","is","an","implementation","of",":","class",":","`","Updater","`","that","uses","multiple","GPUs",".","It","behaves","similarly","to",":","class",":","`","~chainer.training.updaters.StandardUpdater","`",".","The","update","routine","is","modified","to","support","data-parallel","computation","on","multiple","GPUs","in","one","machine",".","It","is","based","on","synchronous","parallel","SGD",":","it","parallelizes","the","gradient","computation","over","a","mini-batch",",","and","updates","the","parameters","only","in","the","main","device",".","Args",":","iterator",":","Dataset","iterator","for","the","training","dataset",".","It","can","also","be","a","dictionary","that","maps","strings","to","iterators",".","If","this","is","just","an","iterator",",","then","the","iterator","is","registered","by","the","name","``","'main","'","``",".","optimizer",":","Optimizer","to","update","parameters",".","It","can","also","be","a","dictionary","that","maps","strings","to","optimizers",".","If","this","is","just","an","optimizer",",","then","the","optimizer","is","registered","by","the","name","``","'main","'","``",".","converter",":","Converter","function","to","build","input","arrays",".","Each","batch","extracted","by","the","main","iterator","is","split","equally","between","the","devices","and","then","passed","with","corresponding","``","device","``","option","to","this","function",".",":","func",":","`","~chainer.dataset.concat_examples","`","is","used","by","default",".","models",":","Dictionary","of","models",".","The","main","model","should","be","the","same","model","attached","to","the","``","'main","'","``","optimizer",".","devices",":","Dictionary","of","devices","to","which","the","training","data","is","sent",".","The","devices","should","be","arranged","in","a","dictionary","with","the","same","structure","as","``","models","``",".","loss_func",":","Loss","function",".","The","model","is","used","as","a","loss","function","by","default",".","loss_scale","(","float",")",":","Loss","scaling","factor",".","Loss","scaling","is","a","useful","technique","to","mitigate","vanishing","gradient","issue","that","tends","to","happen","when","low","precision","data","type","like","float16","is","used","during","training",".","If","you","set","loss","scaling","factor",",","gradients","of","loss","values","are","to","be","multiplied","by","the","factor","before","backprop","starts",".","The","factor","is","propagated","to","whole","gradients","in","a","computational","graph","along","the","backprop",".","The","gradients","of","parameters","are","divided","by","the","factor","just","before","the","parameters","are","to","be","updated",".","auto_new_epoch","(","bool",")",":","If","``","True","``",",",":","meth",":","`","~chainer.Optimizer.new_epoch","`","of","the","main","optimizer","is","automatically","called","when","the","``","is_new_epoch","``","attribute","of","the","main","iterator","is","``","True","``","."]
code: def connect_trainer(self, trainer):
      # Add observers for all (other) models.
      model_main = self.get_optimizer("main").target
      models_others = {
          k: v for k, v in self._models.items() if v != model_main
      }
      for name, model in models_others.items():
          trainer.reporter.add_observer(name, model)
  
file_path: chainer/chainer/training/updaters/parallel_updater.py
imports: import copy
  import six
  import chainer
  from chainer.dataset import convert
  from chainer import function
  from chainer.training.updaters import standard_updater
label_desc: Use image node_id 2 for calling the ParallelUpdater obj's underlying member method code with example usage: obj.connect_trainer(trainer) without return types