class_docstr: The DDP module that supports DataContainer.
  
  This MMDDP has some differences with the original MMDDP:
  
  - It implements a ``forward()`` method
  - It update the code in ``train_step()`` ``val_step()`` and ``_run_ddp_forward()``
  when self.device_ids is None, which means training with CPU
  
  related issues:
  - https://github.com/open-mmlab/mmcv/issues/2524
  - https://github.com/open-mmlab/mmcv/issues/2558
  When the issues above is solved by mmcv, this file can be deleted, then directly
  use the MMDDP implemented by mmcv in MMCVRayEpochRunner.
docstr: train_step() API for module wrapped by DistributedDataParallel.
  
  This method is basically the same as
  ``DistributedDataParallel.forward()``, while replacing
  ``self.module.forward()`` with ``self.module.train_step()``.
  It is compatible with PyTorch 1.1 - 1.5.
docstr_tok: ["train_step","(",")","API","for","module","wrapped","by","DistributedDataParallel",".","This","method","is","basically","the","same","as","``","DistributedDataParallel.forward","(",")","``",",","while","replacing","``","self.module.forward","(",")","``","with","``","self.module.train_step","(",")","``",".","It","is","compatible","with","PyTorch","1.1","-","1.5","."]
code: def train_step(self, *inputs, **kwargs):
      """train_step() API for module wrapped by DistributedDataParallel.
  
      This method is basically the same as
      ``DistributedDataParallel.forward()``, while replacing
      ``self.module.forward()`` with ``self.module.train_step()``.
      It is compatible with PyTorch 1.1 - 1.5.
      """
  
      # In PyTorch >= 1.7, ``reducer._rebuild_buckets()`` is moved from the
      # end of backward to the beginning of forward.
      if (
          "parrots" not in TORCH_VERSION
          and digit_version(TORCH_VERSION) >= digit_version("1.7")
          and self.reducer._rebuild_buckets()
      ):
          print_log(
              "Reducer buckets have been rebuilt in this iteration.",
              logger="mmcv",
          )
  
      if "parrots" not in TORCH_VERSION and digit_version(
          TORCH_VERSION
      ) >= digit_version("1.11.0a0"):
          if self._check_sync_bufs_pre_fwd():
              self._sync_buffers()
      else:
          if (
              getattr(self, "require_forward_param_sync", False)
              and self.require_forward_param_sync
          ):
              self._sync_params()
  
      if self.device_ids:
          inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)
          if len(self.device_ids) == 1:
              output = self.module.train_step(*inputs[0], **kwargs[0])
          else:
              outputs = self.parallel_apply(
                  self._module_copies[: len(inputs)], inputs, kwargs
              )
              output = self.gather(outputs, self.output_device)
      else:
          inputs, kwargs = self.scatter(inputs, kwargs, [-1])
          output = self.module.train_step(*inputs[0], **kwargs[0])
  
      if "parrots" not in TORCH_VERSION and digit_version(
          TORCH_VERSION
      ) >= digit_version("1.11.0a0"):
          if self._check_sync_bufs_post_fwd():
              self._sync_buffers()
  
      if (
          torch.is_grad_enabled()
          and getattr(self, "require_backward_grad_sync", False)
          and self.require_backward_grad_sync
      ):
          if self.find_unused_parameters:
              self.reducer.prepare_for_backward(
                  list(_find_tensors(output))
              )
          else:
              self.reducer.prepare_for_backward([])
      else:
          if "parrots" not in TORCH_VERSION and digit_version(
              TORCH_VERSION
          ) > digit_version("1.2"):
              self.require_forward_param_sync = False
      return output
  
file_path: BigDL/python/orca/src/bigdl/orca/learn/pytorch/experimential/mmcv/distributed.py
imports: from typing import Any, List, Tuple
  import torch
  from torch.nn.parallel.distributed import DistributedDataParallel, _find_tensors
  from mmcv import print_log
  from mmcv.utils import TORCH_VERSION, digit_version
  from mmcv.parallel.scatter_gather import ScatterInputs, scatter_kwargs
label_desc: Use image node_id 4 for calling the MMDistributedDataParallel obj's underlying member method code with example usage: obj.train_step() and returns: output