class_docstr: Implementation of a multiprocess parallel GPU Updater.
  
  This is an implementation of :class:`Updater` that uses multiple GPUs
  with multi-process data parallelism. It uses Nvidia NCCL for communication
  between multiple GPUs.
  
  It behaves similarly to
  :class:`~chainer.training.updaters.StandardUpdater`.
  The update routine is modified to support data-parallel
  computation on multiple GPUs in one machine.
  It is based on synchronous parallel SGD: it
  parallelizes the gradient computation over a mini-batch, and updates the
  parameters only in the main device.
  
  It does not transfer the values collected by :class:`Reporter` in the sub
  devices to the main device. So you can only see the reported values in
  the main device.
  
  Args:
      iterators: List of dataset iterator for the training dataset. The
          number of the iterators must be same to the number of GPUs you use.
      optimizer: Optimizer to update parameters. The model should be attached
          to the optimizer.
      converter: Converter function to build input arrays. Each batch
          extracted by the iterator is split equally between the devices and
          then passed with corresponding ``device`` option to this function.
          :func:`~chainer.dataset.concat_examples` is used by default.
      devices: Dictionary or list of devices to which the training data is
          sent. The master device will be the first one in the list or the
          value attached to the key ``'main'``.
      auto_new_epoch (bool): If ``True``,
          :meth:`~chainer.Optimizer.new_epoch` of the main optimizer is
          automatically called when the ``is_new_epoch`` attribute of the
          main iterator is ``True``.
class_docstr_tok: ["Implementation","of","a","multiprocess","parallel","GPU","Updater",".","This","is","an","implementation","of",":","class",":","`","Updater","`","that","uses","multiple","GPUs","with","multi-process","data","parallelism",".","It","uses","Nvidia","NCCL","for","communication","between","multiple","GPUs",".","It","behaves","similarly","to",":","class",":","`","~chainer.training.updaters.StandardUpdater","`",".","The","update","routine","is","modified","to","support","data-parallel","computation","on","multiple","GPUs","in","one","machine",".","It","is","based","on","synchronous","parallel","SGD",":","it","parallelizes","the","gradient","computation","over","a","mini-batch",",","and","updates","the","parameters","only","in","the","main","device",".","It","does","not","transfer","the","values","collected","by",":","class",":","`","Reporter","`","in","the","sub","devices","to","the","main","device",".","So","you","can","only","see","the","reported","values","in","the","main","device",".","Args",":","iterators",":","List","of","dataset","iterator","for","the","training","dataset",".","The","number","of","the","iterators","must","be","same","to","the","number","of","GPUs","you","use",".","optimizer",":","Optimizer","to","update","parameters",".","The","model","should","be","attached","to","the","optimizer",".","converter",":","Converter","function","to","build","input","arrays",".","Each","batch","extracted","by","the","iterator","is","split","equally","between","the","devices","and","then","passed","with","corresponding","``","device","``","option","to","this","function",".",":","func",":","`","~chainer.dataset.concat_examples","`","is","used","by","default",".","devices",":","Dictionary","or","list","of","devices","to","which","the","training","data","is","sent",".","The","master","device","will","be","the","first","one","in","the","list","or","the","value","attached","to","the","key","``","'main","'","``",".","auto_new_epoch","(","bool",")",":","If","``","True","``",",",":","meth",":","`","~chainer.Optimizer.new_epoch","`","of","the","main","optimizer","is","automatically","called","when","the","``","is_new_epoch","``","attribute","of","the","main","iterator","is","``","True","``","."]
code: def __init__(
      self,
      iterators,
      optimizer,
      converter=convert.concat_examples,
      devices=None,
      auto_new_epoch=True,
  ):
      if not MultiprocessParallelUpdater.available():
          raise Exception(
              "NCCL is not enabled. MultiprocessParallelUpdater "
              "requires NCCL.\n"
              "Please reinstall CuPy after you install NCCL.\n"
              "(see https://docs-cupy.chainer.org/en/latest/install.html)"
          )
      try:
          cuda.cupy.cuda.driver.ctxGetCurrent()
          _cuda_initialized = True
      except cuda.cupy.cuda.driver.CUDADriverError:
          # The context is not initialized, it will be fine.
          _cuda_initialized = False
      if (
          _cuda_initialized
          and multiprocessing.get_start_method()
          not in ("spawn", "forkserver")
      ):
          raise RuntimeError(
              "The CUDA context has been already initialized. "
              "MultiprocessParallelUpdater assumes the context is "
              "uninitialized. Please do not call CUDA API before "
              "MultiprocessParallelUpdater creates processes "
              "or use multiprocessing.set_start_method with "
              "'spawn' or 'forkserver' as arguments."
          )
  
      assert len(iterators) == len(devices)
      for iterator in iterators[1:]:
          assert len(iterator.dataset) == len(iterators[0].dataset)
  
      # Correct optimizer parameters for new minibatch size
      optim = optimizer.__class__.__name__
      if optim in ("Adam", "AdaGrad", "RMSprop"):
          optimizer.eps *= len(devices)
          warnings.warn(
              "optimizer.eps is changed to {} "
              "by MultiprocessParallelUpdater for new batch size.".format(
                  optimizer.eps
              )
          )
      elif optim in ("RMSpropGraves", "AdaDelta"):
          optimizer.eps *= (
              len(devices) ** 2
          )  # not quite right for AdaDelta
          warnings.warn(
              "optimizer.eps is changed to {} "
              "by MultiprocessParallelUpdater for new batch size.".format(
                  optimizer.eps
              )
          )
      elif hasattr(optimizer, "lr"):
          optimizer.lr /= len(devices)
          warnings.warn(
              "optimizer.lr is changed to {} "
              "by MultiprocessParallelUpdater for new batch size.".format(
                  optimizer.lr
              )
          )
  
      super(MultiprocessParallelUpdater, self).__init__(
          iterator=iterators[0],
          optimizer=optimizer,
          converter=converter,
          auto_new_epoch=auto_new_epoch,
      )
  
      if isinstance(devices, dict):
          devices = devices.copy()
          main = devices.pop("main")
          devices = list(six.itervalues(devices))
          devices = [main] + devices
      elif isinstance(devices, (list, tuple)):
          devices = list(devices)
      else:
          raise ValueError(
              "devices argument should be either dict, list or tuple,"
              " but {} was given.".format(type(devices))
          )
  
      if devices is None or any(device is None for device in devices):
          raise ValueError("GPU devices must be specified.")
  
      self._master = optimizer.target
      self._devices = [chainer.get_device(device) for device in devices]
      self._mpu_iterators = iterators
      self._initialized = False
  
      self._pipes = []
      self._workers = []
      self.comm = None
  
imports: import multiprocessing
  import warnings
  import numpy
  import six
  import chainer
  from chainer.backends import cuda
  from chainer.dataset import convert
  from chainer import reporter
  from chainer.training.updaters import standard_updater
label_desc: Use image node_id 1 to create a new MultiprocessParallelUpdater object from inherited base classes: standard_updater with example: obj = MultiprocessParallelUpdater(iterators, optimizer, converter, devices, auto_new_epoch)