class_docstr: A wrapper that is used to replace the DDP module provided by Megatron, which
  is adapted to enable the sophiscated parallel and reduction strategies in
  Fast MoE.
docstr: Keep consitency with Megatron
code: def load_state_dict(self, *args, **kwargs):
      r"""
      Keep consitency with Megatron
      """
      return self.module.load_state_dict(*args, **kwargs)
  
file_path: thu-pacman-faster-moe/fmoe/megatron/distributed.py
imports: from fmoe.distributed import DistributedGroupedDataParallel
label_desc: Use image node_id 4 for calling the DistributedDataParallel obj's underlying member method code with example usage: obj.load_state_dict() and returns: self