class_docstr: Distributed data parallel for ColoTensor. Nested ColoDDP is not supported now.
  
  Example:
      >>> from colossalai.legacy.core import global_context as gpc
      >>> from colossalai.legacy.context import ParallelMode
      >>> model = torch.nn.Linear(20, 1)
      >>> pg = ProcessGroup(tp_degree = world_size//2)
      >>> model = ColoDDP(model, pg)
      >>> logits = model(x)
      >>> loss = criterion(logits, labels)
      >>> model.backward(loss)
  
  Args:
      module (torch.nn.Module): Module to apply DDP.
      process_group (Optional[dist.ProcessGroup], optional): The process group which DDP uses.
          If it's None, the default data parallel group will be used. Defaults to None.
class_docstr_tok: ["Distributed","data","parallel","for","ColoTensor",".","Nested","ColoDDP","is","not","supported","now",".","Example",":",">",">",">","from","colossalai.legacy.core","import","global_context","as","gpc",">",">",">","from","colossalai.legacy.context","import","ParallelMode",">",">",">","model","=","torch.nn.Linear","(","20",",","1",")",">",">",">","pg","=","ProcessGroup","(","tp_degree","=","world_size\/\/2",")",">",">",">","model","=","ColoDDP","(","model",",","pg",")",">",">",">","logits","=","model","(","x",")",">",">",">","loss","=","criterion","(","logits",",","labels",")",">",">",">","model.backward","(","loss",")","Args",":","module","(","torch.nn.Module",")",":","Module","to","apply","DDP",".","process_group","(","Optional","[","dist.ProcessGroup","]",",","optional",")",":","The","process","group","which","DDP","uses",".","If","it","'s","None",",","the","default","data","parallel","group","will","be","used",".","Defaults","to","None","."]
code: def state_dict(self, destination=None, prefix="", keep_vars=False):
      return self.module.state_dict(
          destination=destination, prefix=prefix, keep_vars=keep_vars
      )
  
imports: from collections import OrderedDict
  from functools import partial
  from typing import Iterable, Optional, Set
  import torch
  import torch.distributed
  from colossalai.legacy.tensor import ProcessGroup
  from colossalai.utils import is_ddp_ignored
  from .reducer import Reducer
label_desc: Use image node_id 13 for calling the ColoDDP obj's underlying member method code with example usage: obj.state_dict(destination, prefix, keep_vars) and returns: self