docstr: Compute mean of each element on each processes with NCCL.
  
  The function compute mean of each element in ``sendbuf`` on each
  processes. The result is stored in ``recvbuf``. NCCL is used for
  communication.
  
  Args:
      sendbuf (numpy/cupy array): Input arrays.
      recvbuf (numpy/cupy array): Output arrays.
      n_elems (int): the number of elements in `sendbuf`.
      dtype: Data type of elements used in All-Reduce.
      stream: CUDA stream used for All-Reduce.
docstr_tok: ["Compute","mean","of","each","element","on","each","processes","with","NCCL",".","The","function","compute","mean","of","each","element","in","``","sendbuf","``","on","each","processes",".","The","result","is","stored","in","``","recvbuf","``",".","NCCL","is","used","for","communication",".","Args",":","sendbuf","(","numpy\/cupy","array",")",":","Input","arrays",".","recvbuf","(","numpy\/cupy","array",")",":","Output","arrays",".","n_elems","(","int",")",":","the","number","of","elements","in","`","sendbuf","`",".","dtype",":","Data","type","of","elements","used","in","All-Reduce",".","stream",":","CUDA","stream","used","for","All-Reduce","."]
code: def _multi_node_mean_nccl(
      self, sendbuf, recvbuf, n_elems, dtype, stream=None
  ):
      """Compute mean of each element on each processes with NCCL.
  
      The function compute mean of each element in ``sendbuf`` on each
      processes. The result is stored in ``recvbuf``. NCCL is used for
      communication.
  
      Args:
          sendbuf (numpy/cupy array): Input arrays.
          recvbuf (numpy/cupy array): Output arrays.
          n_elems (int): the number of elements in `sendbuf`.
          dtype: Data type of elements used in All-Reduce.
          stream: CUDA stream used for All-Reduce.
  
      """
      if chainer.is_debug():
          stream.synchronize()
          array_a = sendbuf.array(n_elems, dtype=dtype)
          array_b = recvbuf.array(n_elems, dtype=dtype)
          self._check_ready_to_allreduce(array_a, array_b)
  
      if stream is None:
          stream = chainer.cuda.Stream.null
      self._init_comms()
      type_id = _communication_utility._get_nccl_type_id(dtype)
      self.nccl_comm.allReduce(
          sendbuf.ptr(),
          recvbuf.ptr(),
          n_elems,
          type_id,
          nccl.NCCL_SUM,
          stream.ptr,
      )
      div_by_size = chainer.cuda.elementwise(
          "",
          "{} x".format(dtype.name),
          "x *= (1.0/{})".format(self.size),
          "div_by_size",
      )
      div_by_size(recvbuf.array(n_elems, dtype=dtype), stream=stream)
  
      if chainer.is_debug():
          stream.synchronize()
          self._ensure_all_finite(recvbuf.array(n_elems, dtype=dtype))
  
file_path: chainer/chainermn/communicators/pure_nccl_communicator.py
imports: import warnings
  import chainer.cuda
  from chainermn.communicators import _communication_utility
  from chainermn.communicators import _memory_utility
  from chainermn.communicators import mpi_communicator_base
  from chainermn import nccl
  import numpy
label_desc: Use image node_id 10 for calling the PureNcclCommunicator obj's underlying member method code with example usage: obj._multi_node_mean_nccl(sendbuf, recvbuf, n_elems, dtype, stream) without return types