class_docstr: ``OneflowDDPDriver`` 实现了动态图下使用 ``DistributedDataParallel`` 进行的数据并行分布式训练。
  
  .. note::
  
      您在绝大多数情况下不需要自己使用到该类，通过向 ``Trainer`` 传入正确的参数，您可以方便快速地部署您的分布式训练。
  
      ``OneflowDDPDriver`` 目前支持两种启动方式：
      
          1. 用户不做任何处理，通过运行 ``python -m oneflow.distributed.launch --nproc_per_node 2 train.py`` 启动；
          2. 用户将模型通过 ``DistributedDataParallel`` 处理后，通过运行 ``python -m oneflow.distributed.launch --nproc_per_node 2 train.py`` 启动；
  
      注意多机的启动强制要求用户在每一台机器上使用 ``python -m oneflow.distributed.launch`` 启动；因此我们不会在 ``OneflowDDPDriver`` 中保存
      任何当前有多少台机器的信息。
  
  :param model: 传入给 ``Trainer`` 的 ``model`` 参数
  :param parallel_device: 该参数无效，**fastNLP** 会自动获取当前进程的设备
  :param fp16: 是否开启 fp16 训练；目前该参数无效
  :param oneflow_kwargs: 
      * *ddp_kwargs* -- 用于 ``DistributedDataParallel`` 的其它参数，详情可查阅 **oneflow** 的官方文档
  :kwargs:
      * *model_wo_auto_param_call* (``bool``) -- 是否关闭在训练时调用我们的 ``auto_param_call`` 函数来自动匹配 batch 和前向函数的参数的行为
  
      .. note::
  
          关于该参数的详细说明，请参见 :class:`~fastNLP.core.controllers.Trainer` 中的描述；函数 ``auto_param_call`` 详见 :func:`fastNLP.core.utils.auto_param_call`。
class_docstr_tok: ["``","OneflowDDPDriver","``","\u5b9e\u73b0\u4e86\u52a8\u6001\u56fe\u4e0b\u4f7f\u7528","``","DistributedDataParallel","``","\u8fdb\u884c\u7684\u6570\u636e\u5e76\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u3002","..","note",":",":","\u60a8\u5728\u7edd\u5927\u591a\u6570\u60c5\u51b5\u4e0b\u4e0d\u9700\u8981\u81ea\u5df1\u4f7f\u7528\u5230\u8be5\u7c7b\uff0c\u901a\u8fc7\u5411","``","Trainer","``","\u4f20\u5165\u6b63\u786e\u7684\u53c2\u6570\uff0c\u60a8\u53ef\u4ee5\u65b9\u4fbf\u5feb\u901f\u5730\u90e8\u7f72\u60a8\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3\u3002","``","OneflowDDPDriver","``","\u76ee\u524d\u652f\u6301\u4e24\u79cd\u542f\u52a8\u65b9\u5f0f\uff1a","1",".","\u7528\u6237\u4e0d\u505a\u4efb\u4f55\u5904\u7406\uff0c\u901a\u8fc7\u8fd0\u884c","``","python","-m","oneflow.distributed.launch","--","nproc_per_node","2","train.py","``","\u542f\u52a8\uff1b","2",".","\u7528\u6237\u5c06\u6a21\u578b\u901a\u8fc7","``","DistributedDataParallel","``","\u5904\u7406\u540e\uff0c\u901a\u8fc7\u8fd0\u884c","``","python","-m","oneflow.distributed.launch","--","nproc_per_node","2","train.py","``","\u542f\u52a8\uff1b","\u6ce8\u610f\u591a\u673a\u7684\u542f\u52a8\u5f3a\u5236\u8981\u6c42\u7528\u6237\u5728\u6bcf\u4e00\u53f0\u673a\u5668\u4e0a\u4f7f\u7528","``","python","-m","oneflow.distributed.launch","``","\u542f\u52a8\uff1b\u56e0\u6b64\u6211\u4eec\u4e0d\u4f1a\u5728","``","OneflowDDPDriver","``","\u4e2d\u4fdd\u5b58","\u4efb\u4f55\u5f53\u524d\u6709\u591a\u5c11\u53f0\u673a\u5668\u7684\u4fe1\u606f\u3002",":","param","model",":","\u4f20\u5165\u7ed9","``","Trainer","``","\u7684","``","model","``","\u53c2\u6570",":","param","parallel_device",":","\u8be5\u53c2\u6570\u65e0\u6548\uff0c","*","*","fastNLP","*","*","\u4f1a\u81ea\u52a8\u83b7\u53d6\u5f53\u524d\u8fdb\u7a0b\u7684\u8bbe\u5907",":","param","fp16",":","\u662f\u5426\u5f00\u542f","fp16","\u8bad\u7ec3\uff1b\u76ee\u524d\u8be5\u53c2\u6570\u65e0\u6548",":","param","oneflow_kwargs",":","*","*","ddp_kwargs","*","--","\u7528\u4e8e","``","DistributedDataParallel","``","\u7684\u5176\u5b83\u53c2\u6570\uff0c\u8be6\u60c5\u53ef\u67e5\u9605","*","*","oneflow","*","*","\u7684\u5b98\u65b9\u6587\u6863",":","kwargs",":","*","*","model_wo_auto_param_call","*","(","``","bool","``",")","--","\u662f\u5426\u5173\u95ed\u5728\u8bad\u7ec3\u65f6\u8c03\u7528\u6211\u4eec\u7684","``","auto_param_call","``","\u51fd\u6570\u6765\u81ea\u52a8\u5339\u914d","batch","\u548c\u524d\u5411\u51fd\u6570\u7684\u53c2\u6570\u7684\u884c\u4e3a","..","note",":",":","\u5173\u4e8e\u8be5\u53c2\u6570\u7684\u8be6\u7ec6\u8bf4\u660e\uff0c\u8bf7\u53c2\u89c1",":","class",":","`","~fastNLP.core.controllers.Trainer","`","\u4e2d\u7684\u63cf\u8ff0\uff1b\u51fd\u6570","``","auto_param_call","``","\u8be6\u89c1",":","func",":","`","fastNLP.core.utils.auto_param_call","`","\u3002"]
docstr: 将 ``obj`` 互相传送到其它所有的 rank 上，其中 ``obj`` 可能是 Tensor，也可能是嵌套结构的 object 。如果不是基础类型的数据，将会尝试通过
  pickle 进行序列化，接收到之后再反序列化。
  
  example::
  
      >>> # rank 0
      >>> obj = {'a': 1, 'b':[1, 2], 'c':{'d': 1}}
      >>> # rank 1
      >>> obj = {'a': 1, 'b':[1, 2], 'c':{'d': 2}}
      >>> # after all_gather():
      >>> result = [
              {'a': 1, 'b':[1, 2], 'c':{'d': 1}},
              {'a': 1, 'b':[1, 2], 'c':{'d': 2}}
          ]
  
  :param obj: 需要传输的对象，在每个 rank 上都应该保持相同的结构。
  :param group: 该参数无效。
  :return: 所有 rank 发送的 ``obj`` 聚合在一起的内容；如果环境变量 ``FASTNLP_NO_SYNC`` 为 **2** 则不会执行，直接返回 ``[obj]`` 。
docstr_tok: ["\u5c06","``","obj","``","\u4e92\u76f8\u4f20\u9001\u5230\u5176\u5b83\u6240\u6709\u7684","rank","\u4e0a\uff0c\u5176\u4e2d","``","obj","``","\u53ef\u80fd\u662f","Tensor\uff0c\u4e5f\u53ef\u80fd\u662f\u5d4c\u5957\u7ed3\u6784\u7684","object","\u3002\u5982\u679c\u4e0d\u662f\u57fa\u7840\u7c7b\u578b\u7684\u6570\u636e\uff0c\u5c06\u4f1a\u5c1d\u8bd5\u901a\u8fc7","pickle","\u8fdb\u884c\u5e8f\u5217\u5316\uff0c\u63a5\u6536\u5230\u4e4b\u540e\u518d\u53cd\u5e8f\u5217\u5316\u3002","example",":",":",">",">",">","#","rank","0",">",">",">","obj","=","{","'","a","'",":","1",",","'","b","'",":","[","1",",","2","]",",","'","c","'",":","{","'d","'",":","1","}","}",">",">",">","#","rank","1",">",">",">","obj","=","{","'","a","'",":","1",",","'","b","'",":","[","1",",","2","]",",","'","c","'",":","{","'d","'",":","2","}","}",">",">",">","#","after","all_gather","(",")",":",">",">",">","result","=","[","{","'","a","'",":","1",",","'","b","'",":","[","1",",","2","]",",","'","c","'",":","{","'d","'",":","1","}","}",",","{","'","a","'",":","1",",","'","b","'",":","[","1",",","2","]",",","'","c","'",":","{","'d","'",":","2","}","}","]",":","param","obj",":","\u9700\u8981\u4f20\u8f93\u7684\u5bf9\u8c61\uff0c\u5728\u6bcf\u4e2a","rank","\u4e0a\u90fd\u5e94\u8be5\u4fdd\u6301\u76f8\u540c\u7684\u7ed3\u6784\u3002",":","param","group",":","\u8be5\u53c2\u6570\u65e0\u6548\u3002",":","return",":","\u6240\u6709","rank","\u53d1\u9001\u7684","``","obj","``","\u805a\u5408\u5728\u4e00\u8d77\u7684\u5185\u5bb9\uff1b\u5982\u679c\u73af\u5883\u53d8\u91cf","``","FASTNLP_NO_SYNC","``","\u4e3a","*","*","2","*","*","\u5219\u4e0d\u4f1a\u6267\u884c\uff0c\u76f4\u63a5\u8fd4\u56de","``","[","obj","]","``","\u3002"]
code: def all_gather(self, obj) -> List:
      r"""
      将 ``obj`` 互相传送到其它所有的 rank 上，其中 ``obj`` 可能是 Tensor，也可能是嵌套结构的 object 。如果不是基础类型的数据，将会尝试通过
      pickle 进行序列化，接收到之后再反序列化。
  
      example::
  
          >>> # rank 0
          >>> obj = {'a': 1, 'b':[1, 2], 'c':{'d': 1}}
          >>> # rank 1
          >>> obj = {'a': 1, 'b':[1, 2], 'c':{'d': 2}}
          >>> # after all_gather():
          >>> result = [
                  {'a': 1, 'b':[1, 2], 'c':{'d': 1}},
                  {'a': 1, 'b':[1, 2], 'c':{'d': 2}}
              ]
  
      :param obj: 需要传输的对象，在每个 rank 上都应该保持相同的结构。
      :param group: 该参数无效。
      :return: 所有 rank 发送的 ``obj`` 聚合在一起的内容；如果环境变量 ``FASTNLP_NO_SYNC`` 为 **2** 则不会执行，直接返回 ``[obj]`` 。
      """
      if (
          int(os.environ.get(FASTNLP_NO_SYNC, 0)) == 2
      ):  # 如果 FASTNLP_NO_SYNC 表示不执行
          return [obj]
      return fastnlp_oneflow_all_gather(obj)
  
imports: import os
  from typing import List, Optional, Union, Dict
  from fastNLP.envs.imports import _NEED_IMPORT_ONEFLOW
  from .oneflow_driver import OneflowDriver
  from fastNLP.core.drivers.oneflow_driver.utils import replace_sampler, replace_batch_sampler
  from fastNLP.core.utils import check_user_specific_params
  from fastNLP.core.samplers import ReproducibleSampler, RandomSampler, UnrepeatedSequentialSampler, ReproducibleBatchSampler, re_instantiate_sampler, UnrepeatedSampler, conversion_between_reproducible_and_unrepeated_sampler
  from fastNLP.envs import FASTNLP_GLOBAL_SEED, FASTNLP_NO_SYNC
  from fastNLP.core.log import logger
  from fastNLP.core.drivers.oneflow_driver.dist_utils import fastnlp_oneflow_all_gather, fastnlp_oneflow_broadcast_object
  from .utils import _check_dataloader_args_for_distributed
label_desc: Use image node_id 20 for calling the OneflowDDPDriver obj's underlying member method code with example usage: obj.all_gather(obj) and returns: fastnlp_oneflow_all_gather, list