class_docstr: ``PaddleDataLoader`` 是专门提供给 ``paddle`` 框架的 ``DataLoader`` ,其集成了 ``fastNLP`` 的 ``Collator`` ，
  具体详见 :class:`~fastNLP.core.collators.Collator`， 并对 ``paddle`` 的 ``DataLoader`` 进行了
  封装，使得其具备以下功能：
  
  1. ``PaddleDataLoader`` 支持输入的 dataset 是无框架的，只要实现了 __getitem__() 和 __len__() 的对象即可，
     当不使用  :class:`~fastNLP.core.dataset.DataSet` 时也不需要传入 collate_fn, 只要只需要将 ``collate_fn='auto'`` 就能够自动
     探测数据的类型并判断能否 pad 。此时可以调用 ``set_pad`` 和 ``set_ignore`` 方法来设置 field 的 pad_val 或者忽略某个 field 的 pad 操作。
  
      Example::
  
          from fastNLP import PaddleDataLoader
          class MyDataset:
              def __init(self, data_lens=100):
                  self.data_lens = 100
              def __getitem__(self, item):
                  if item % 2 == 0:
                      return {'x':[101, 256, 453], 'y': 0}
                  else:
                      return {'x': [101, 200], 'y': 1}
              def __len__(self):
                  return self.data_lens
          dataset = MyDataset()
          paddle_dl = PaddleDataLoader(dataset, collate_fn='auto')
          for batch in paddle_dl:
              ...
  
  2.当 collate_fn 为 ``None`` 时，``PaddleDataLoader`` 默认使用 ``paddle`` 自带的 ``default_collate_fn`` 作为 collate_fn 的值
  
      .. note::
          当传入的dataset为fastNLP的DataSet时，collate_fn不能为None。默认可以是"auto"或者自定义callable函数。
  
  3. 当 collate_fn 为 :class:`Callable` 时，该 Callable 函数应当接受一个 batch 参数作为输入， batch 是一个 List 对象且 List 中的每一条数据都是
     dataset 的一条数据；该 Callable 函数还应当返回一个对象。
  
  :param dataset: 实现了 __getitem__() 和 __len__() 的对象。
  :param feed_list: feed Tensor list.
      这个张量能被 ``paddle.static.data`` 创建。 如果 :attr:`return_list` 是 ``False``, 那么 :attr:`feed_list`
      应该被设置。 默认为 ``None `` 。
  :param places: 将数据放进的一个 list 的 place。 :attr:`places` 能为 None.
      如果 :attr:`places` 为 None， 默认放在 CPUPlace 或者 CUDAPlace(0) 设备上。 如果 ``places`` 是一个 list 类型的 字符串， 那么字符串
      可以是 ``cpu`` , ``gpu:x`` 或者 ``gpu_pinned`` ， 其中 ``x`` 是 gpu 的下标。
  :param return_list: 每个设备上的返回值是否为以列表形式显示。 如果 :attr:`return_list=False`,
      每个设备上的返回值值为 str -> Tensor 的 dict， 其中 dict 的 key 为每个 fed Tensors 的名字。
      如果 :attr:`return_list=True`， 每个设备上的返回值值为 list(Tensor)。 :attr:`return_list` 只能在动态图情况下设置为 ``True`` .
      默认值为 ``True`` 。
  :param batch_sampler: 实现了 __len__() 和 __iter__() 的实例化对象，，其__iter__() 方法每次都会返回一个 List 对象， List中的值为
      dataset 的下标 index ；默认为 ``None``，当其不为 ``None`` 时，``bacth_size``, ``shuffle`` 参数均失效。
  :param batch_size: 批次大小，默认为 ``16`` 且当 ``batch_sampler`` 为 None 有效。
  :param shuffle: 是否打乱数据集， 默认为 ``None``, 如果传入的 ``ds_or_db`` 可以判断出哪个是 ``'train'`` 则设置其 shuffle 为 ``True`` ，
      其它的为 False 。
  :param drop_last: 当 ``drop_last=True`` 时，``PaddleDataLoader`` 会扔掉最后一个长度小于 ``batch_size`` 的 batch 数据;
      若 ``drop_last=False`` , 则会返回该 batch 数据。 默认为 ``False`` 。
  :param collate_fn: 用于从 dataset 取到的一个 batch 数据进行打包处理的 Callable 函数，其值应该为以下三个: ``[None, "auto", Callable]``.
  
      * callate_fn 为 ``None`` 时，需要注意的是此时传进来的 datset 类型不能为 :class:`~fastNLP.core.dataset.DataSet` , 当 collate_fn 为 ``None`` 时，
      ``PaddleDataLoader`` 调用默认的 Paddle 框架的 ``DataLoader`` 自带的 `default_collate_fn` 作为 callate_fn 的默认值， 其无法处理
      :class:`~fastNLP.core.dataset.DataSet` 的dataset对象。
      * callate_fn 为 ``'auto'`` 时，``PaddleDataLoader`` 使用 :class:`~fastNLP.core.collators.Collator` 作为 collate_fn 的默认值。
      此时可以配套使用 ``PaddleDataLoader`` 的 ``set_pad`` 和 ``set_ignore`` 方法来设置 pad_val 或 忽略某个 field 的检测。
      * collate_fn 为 :class:`Callable` 时， 该 Callable 函数应当接受一个 batch 参数作为输入， batch 是一个 List 对象且 List 中的每一条数据都是
      dataset 的一条数据；该 Callable 函数还应当返回一个对象。
  
  :param num_workers: 当 ``num_workers > 0`` 时, ``PaddleDataLoader`` 会开启 ``num_workers`` 个子进程来处理数据， 可以加快
      数据处理速度，但同时也消耗大量内存。 当 ``num_workers=0`` 时， 不开启子进程。 默认为 ``0``。
  :param use_buffer_reader: 是否开启 buffer_reader 。如果 ``use_buffer_reader=True`` ，那么 ``PaddleDataLoader`` 会异步地预取下一个 batch 的
      数据，因此它将会加快数据传输的速度，但是将会占用更多的内存或者显存。默认值是 ``True``。
  :param use_shared_memory: 是否使用共享内存。当 ``use_shared_memory=True`` 时，将采用共享内存来加快将数据放进进程队列。建议仅当计算机上的
      共享空间足够大时。（例如 Linux 上的 /dev/shm/ 空间足够大）共享内存仅在多进程模式（ ``num_workers>0`` ）下生效。
  :param timeout: 从子进程的输出队列获取数据的超时值
  :param worker_init_fn: init 函数，如果不设置为 None ,则将会在每个子进程初始化时调用该函数。
  :param persistent_workers: 如果其为 ``True``, ``PaddleDataLoader`` 在迭代完一次 dataset 后不会关闭所有进程。默认为 ``False`` 
class_docstr_tok: ["``","PaddleDataLoader","``","\u662f\u4e13\u95e8\u63d0\u4f9b\u7ed9","``","paddle","``","\u6846\u67b6\u7684","``","DataLoader","``",",","\u5176\u96c6\u6210\u4e86","``","fastNLP","``","\u7684","``","Collator","``","\uff0c","\u5177\u4f53\u8be6\u89c1",":","class",":","`","~fastNLP.core.collators.Collator","`","\uff0c","\u5e76\u5bf9","``","paddle","``","\u7684","``","DataLoader","``","\u8fdb\u884c\u4e86","\u5c01\u88c5\uff0c\u4f7f\u5f97\u5176\u5177\u5907\u4ee5\u4e0b\u529f\u80fd\uff1a","1",".","``","PaddleDataLoader","``","\u652f\u6301\u8f93\u5165\u7684","dataset","\u662f\u65e0\u6846\u67b6\u7684\uff0c\u53ea\u8981\u5b9e\u73b0\u4e86","__getitem__","(",")","\u548c","__len__","(",")","\u7684\u5bf9\u8c61\u5373\u53ef\uff0c","\u5f53\u4e0d\u4f7f\u7528",":","class",":","`","~fastNLP.core.dataset.DataSet","`","\u65f6\u4e5f\u4e0d\u9700\u8981\u4f20\u5165","collate_fn",",","\u53ea\u8981\u53ea\u9700\u8981\u5c06","``","collate_fn='auto","'","``","\u5c31\u80fd\u591f\u81ea\u52a8","\u63a2\u6d4b\u6570\u636e\u7684\u7c7b\u578b\u5e76\u5224\u65ad\u80fd\u5426","pad","\u3002\u6b64\u65f6\u53ef\u4ee5\u8c03\u7528","``","set_pad","``","\u548c","``","set_ignore","``","\u65b9\u6cd5\u6765\u8bbe\u7f6e","field","\u7684","pad_val","\u6216\u8005\u5ffd\u7565\u67d0\u4e2a","field","\u7684","pad","\u64cd\u4f5c\u3002","Example",":",":","from","fastNLP","import","PaddleDataLoader","class","MyDataset",":","def","__init","(","self",",","data_lens=100",")",":","self.data_lens","=","100","def","__getitem__","(","self",",","item",")",":","if","item","%","2","==","0",":","return","{","'","x","'",":","[","101",",","256",",","453","]",",","'","y","'",":","0","}","else",":","return","{","'","x","'",":","[","101",",","200","]",",","'","y","'",":","1","}","def","__len__","(","self",")",":","return","self.data_lens","dataset","=","MyDataset","(",")","paddle_dl","=","PaddleDataLoader","(","dataset",",","collate_fn='auto","'",")","for","batch","in","paddle_dl",":","...","2.\u5f53","collate_fn","\u4e3a","``","None","``","\u65f6\uff0c","``","PaddleDataLoader","``","\u9ed8\u8ba4\u4f7f\u7528","``","paddle","``","\u81ea\u5e26\u7684","``","default_collate_fn","``","\u4f5c\u4e3a","collate_fn","\u7684\u503c","..","note",":",":","\u5f53\u4f20\u5165\u7684dataset\u4e3afastNLP\u7684DataSet\u65f6\uff0ccollate_fn\u4e0d\u80fd\u4e3aNone\u3002\u9ed8\u8ba4\u53ef\u4ee5\u662f","''","auto","''","\u6216\u8005\u81ea\u5b9a\u4e49callable\u51fd\u6570\u3002","3",".","\u5f53","collate_fn","\u4e3a",":","class",":","`","Callable","`","\u65f6\uff0c\u8be5","Callable","\u51fd\u6570\u5e94\u5f53\u63a5\u53d7\u4e00\u4e2a","batch","\u53c2\u6570\u4f5c\u4e3a\u8f93\u5165\uff0c","batch","\u662f\u4e00\u4e2a","List","\u5bf9\u8c61\u4e14","List","\u4e2d\u7684\u6bcf\u4e00\u6761\u6570\u636e\u90fd\u662f","dataset","\u7684\u4e00\u6761\u6570\u636e\uff1b\u8be5","Callable","\u51fd\u6570\u8fd8\u5e94\u5f53\u8fd4\u56de\u4e00\u4e2a\u5bf9\u8c61\u3002",":","param","dataset",":","\u5b9e\u73b0\u4e86","__getitem__","(",")","\u548c","__len__","(",")","\u7684\u5bf9\u8c61\u3002",":","param","feed_list",":","feed","Tensor","list",".","\u8fd9\u4e2a\u5f20\u91cf\u80fd\u88ab","``","paddle.static.data","``","\u521b\u5efa\u3002","\u5982\u679c",":","attr",":","`","return_list","`","\u662f","``","False","``",",","\u90a3\u4e48",":","attr",":","`","feed_list","`","\u5e94\u8be5\u88ab\u8bbe\u7f6e\u3002","\u9ed8\u8ba4\u4e3a","``","None","``","\u3002",":","param","places",":","\u5c06\u6570\u636e\u653e\u8fdb\u7684\u4e00\u4e2a","list","\u7684","place\u3002",":","attr",":","`","places","`","\u80fd\u4e3a","None",".","\u5982\u679c",":","attr",":","`","places","`","\u4e3a","None\uff0c","\u9ed8\u8ba4\u653e\u5728","CPUPlace","\u6216\u8005","CUDAPlace","(","0",")","\u8bbe\u5907\u4e0a\u3002","\u5982\u679c","``","places","``","\u662f\u4e00\u4e2a","list","\u7c7b\u578b\u7684","\u5b57\u7b26\u4e32\uff0c","\u90a3\u4e48\u5b57\u7b26\u4e32","\u53ef\u4ee5\u662f","``","cpu","``",",","``","gpu",":","x","``","\u6216\u8005","``","gpu_pinned","``","\uff0c","\u5176\u4e2d","``","x","``","\u662f","gpu","\u7684\u4e0b\u6807\u3002",":","param","return_list",":","\u6bcf\u4e2a\u8bbe\u5907\u4e0a\u7684\u8fd4\u56de\u503c\u662f\u5426\u4e3a\u4ee5\u5217\u8868\u5f62\u5f0f\u663e\u793a\u3002","\u5982\u679c",":","attr",":","`","return_list=False","`",",","\u6bcf\u4e2a\u8bbe\u5907\u4e0a\u7684\u8fd4\u56de\u503c\u503c\u4e3a","str","-",">","Tensor","\u7684","dict\uff0c","\u5176\u4e2d","dict","\u7684","key","\u4e3a\u6bcf\u4e2a","fed","Tensors","\u7684\u540d\u5b57\u3002","\u5982\u679c",":","attr",":","`","return_list=True","`","\uff0c","\u6bcf\u4e2a\u8bbe\u5907\u4e0a\u7684\u8fd4\u56de\u503c\u503c\u4e3a","list","(","Tensor",")","\u3002",":","attr",":","`","return_list","`","\u53ea\u80fd\u5728\u52a8\u6001\u56fe\u60c5\u51b5\u4e0b\u8bbe\u7f6e\u4e3a","``","True","``",".","\u9ed8\u8ba4\u503c\u4e3a","``","True","``","\u3002",":","param","batch_sampler",":","\u5b9e\u73b0\u4e86","__len__","(",")","\u548c","__iter__","(",")","\u7684\u5b9e\u4f8b\u5316\u5bf9\u8c61\uff0c\uff0c\u5176__iter__","(",")","\u65b9\u6cd5\u6bcf\u6b21\u90fd\u4f1a\u8fd4\u56de\u4e00\u4e2a","List","\u5bf9\u8c61\uff0c","List\u4e2d\u7684\u503c\u4e3a","dataset","\u7684\u4e0b\u6807","index","\uff1b\u9ed8\u8ba4\u4e3a","``","None","``","\uff0c\u5f53\u5176\u4e0d\u4e3a","``","None","``","\u65f6\uff0c","``","bacth_size","``",",","``","shuffle","``","\u53c2\u6570\u5747\u5931\u6548\u3002",":","param","batch_size",":","\u6279\u6b21\u5927\u5c0f\uff0c\u9ed8\u8ba4\u4e3a","``","16","``","\u4e14\u5f53","``","batch_sampler","``","\u4e3a","None","\u6709\u6548\u3002",":","param","shuffle",":","\u662f\u5426\u6253\u4e71\u6570\u636e\u96c6\uff0c","\u9ed8\u8ba4\u4e3a","``","None","``",",","\u5982\u679c\u4f20\u5165\u7684","``","ds_or_db","``","\u53ef\u4ee5\u5224\u65ad\u51fa\u54ea\u4e2a\u662f","``","'train","'","``","\u5219\u8bbe\u7f6e\u5176","shuffle","\u4e3a","``","True","``","\uff0c","\u5176\u5b83\u7684\u4e3a","False","\u3002",":","param","drop_last",":","\u5f53","``","drop_last=True","``","\u65f6\uff0c","``","PaddleDataLoader","``","\u4f1a\u6254\u6389\u6700\u540e\u4e00\u4e2a\u957f\u5ea6\u5c0f\u4e8e","``","batch_size","``","\u7684","batch","\u6570\u636e",";","\u82e5","``","drop_last=False","``",",","\u5219\u4f1a\u8fd4\u56de\u8be5","batch","\u6570\u636e\u3002","\u9ed8\u8ba4\u4e3a","``","False","``","\u3002",":","param","collate_fn",":","\u7528\u4e8e\u4ece","dataset","\u53d6\u5230\u7684\u4e00\u4e2a","batch","\u6570\u636e\u8fdb\u884c\u6253\u5305\u5904\u7406\u7684","Callable","\u51fd\u6570\uff0c\u5176\u503c\u5e94\u8be5\u4e3a\u4ee5\u4e0b\u4e09\u4e2a",":","``","[","None",",","``","auto","''",",","Callable","]","``",".","*","callate_fn","\u4e3a","``","None","``","\u65f6\uff0c\u9700\u8981\u6ce8\u610f\u7684\u662f\u6b64\u65f6\u4f20\u8fdb\u6765\u7684","datset","\u7c7b\u578b\u4e0d\u80fd\u4e3a",":","class",":","`","~fastNLP.core.dataset.DataSet","`",",","\u5f53","collate_fn","\u4e3a","``","None","``","\u65f6\uff0c","``","PaddleDataLoader","``","\u8c03\u7528\u9ed8\u8ba4\u7684","Paddle","\u6846\u67b6\u7684","``","DataLoader","``","\u81ea\u5e26\u7684","`","default_collate_fn","`","\u4f5c\u4e3a","callate_fn","\u7684\u9ed8\u8ba4\u503c\uff0c","\u5176\u65e0\u6cd5\u5904\u7406",":","class",":","`","~fastNLP.core.dataset.DataSet","`","\u7684dataset\u5bf9\u8c61\u3002","*","callate_fn","\u4e3a","``","'auto","'","``","\u65f6\uff0c","``","PaddleDataLoader","``","\u4f7f\u7528",":","class",":","`","~fastNLP.core.collators.Collator","`","\u4f5c\u4e3a","collate_fn","\u7684\u9ed8\u8ba4\u503c\u3002","\u6b64\u65f6\u53ef\u4ee5\u914d\u5957\u4f7f\u7528","``","PaddleDataLoader","``","\u7684","``","set_pad","``","\u548c","``","set_ignore","``","\u65b9\u6cd5\u6765\u8bbe\u7f6e","pad_val","\u6216","\u5ffd\u7565\u67d0\u4e2a","field","\u7684\u68c0\u6d4b\u3002","*","collate_fn","\u4e3a",":","class",":","`","Callable","`","\u65f6\uff0c","\u8be5","Callable","\u51fd\u6570\u5e94\u5f53\u63a5\u53d7\u4e00\u4e2a","batch","\u53c2\u6570\u4f5c\u4e3a\u8f93\u5165\uff0c","batch","\u662f\u4e00\u4e2a","List","\u5bf9\u8c61\u4e14","List","\u4e2d\u7684\u6bcf\u4e00\u6761\u6570\u636e\u90fd\u662f","dataset","\u7684\u4e00\u6761\u6570\u636e\uff1b\u8be5","Callable","\u51fd\u6570\u8fd8\u5e94\u5f53\u8fd4\u56de\u4e00\u4e2a\u5bf9\u8c61\u3002",":","param","num_workers",":","\u5f53","``","num_workers",">","0","``","\u65f6",",","``","PaddleDataLoader","``","\u4f1a\u5f00\u542f","``","num_workers","``","\u4e2a\u5b50\u8fdb\u7a0b\u6765\u5904\u7406\u6570\u636e\uff0c","\u53ef\u4ee5\u52a0\u5feb","\u6570\u636e\u5904\u7406\u901f\u5ea6\uff0c\u4f46\u540c\u65f6\u4e5f\u6d88\u8017\u5927\u91cf\u5185\u5b58\u3002","\u5f53","``","num_workers=0","``","\u65f6\uff0c","\u4e0d\u5f00\u542f\u5b50\u8fdb\u7a0b\u3002","\u9ed8\u8ba4\u4e3a","``","0","``","\u3002",":","param","use_buffer_reader",":","\u662f\u5426\u5f00\u542f","buffer_reader","\u3002\u5982\u679c","``","use_buffer_reader=True","``","\uff0c\u90a3\u4e48","``","PaddleDataLoader","``","\u4f1a\u5f02\u6b65\u5730\u9884\u53d6\u4e0b\u4e00\u4e2a","batch","\u7684","\u6570\u636e\uff0c\u56e0\u6b64\u5b83\u5c06\u4f1a\u52a0\u5feb\u6570\u636e\u4f20\u8f93\u7684\u901f\u5ea6\uff0c\u4f46\u662f\u5c06\u4f1a\u5360\u7528\u66f4\u591a\u7684\u5185\u5b58\u6216\u8005\u663e\u5b58\u3002\u9ed8\u8ba4\u503c\u662f","``","True","``","\u3002",":","param","use_shared_memory",":","\u662f\u5426\u4f7f\u7528\u5171\u4eab\u5185\u5b58\u3002\u5f53","``","use_shared_memory=True","``","\u65f6\uff0c\u5c06\u91c7\u7528\u5171\u4eab\u5185\u5b58\u6765\u52a0\u5feb\u5c06\u6570\u636e\u653e\u8fdb\u8fdb\u7a0b\u961f\u5217\u3002\u5efa\u8bae\u4ec5\u5f53\u8ba1\u7b97\u673a\u4e0a\u7684","\u5171\u4eab\u7a7a\u95f4\u8db3\u591f\u5927\u65f6\u3002\uff08\u4f8b\u5982","Linux","\u4e0a\u7684","\/dev\/shm\/","\u7a7a\u95f4\u8db3\u591f\u5927\uff09\u5171\u4eab\u5185\u5b58\u4ec5\u5728\u591a\u8fdb\u7a0b\u6a21\u5f0f\uff08","``","num_workers",">","0","``","\uff09\u4e0b\u751f\u6548\u3002",":","param","timeout",":","\u4ece\u5b50\u8fdb\u7a0b\u7684\u8f93\u51fa\u961f\u5217\u83b7\u53d6\u6570\u636e\u7684\u8d85\u65f6\u503c",":","param","worker_init_fn",":","init","\u51fd\u6570\uff0c\u5982\u679c\u4e0d\u8bbe\u7f6e\u4e3a","None",",","\u5219\u5c06\u4f1a\u5728\u6bcf\u4e2a\u5b50\u8fdb\u7a0b\u521d\u59cb\u5316\u65f6\u8c03\u7528\u8be5\u51fd\u6570\u3002",":","param","persistent_workers",":","\u5982\u679c\u5176\u4e3a","``","True","``",",","``","PaddleDataLoader","``","\u5728\u8fed\u4ee3\u5b8c\u4e00\u6b21","dataset","\u540e\u4e0d\u4f1a\u5173\u95ed\u6240\u6709\u8fdb\u7a0b\u3002\u9ed8\u8ba4\u4e3a","``","False","``"]
docstr: 如果 collate_fn 是 Collator 对象，得到该对象。如果没有的话，返回 None
  
  :return:
docstr_tok: ["\u5982\u679c","collate_fn","\u662f","Collator","\u5bf9\u8c61\uff0c\u5f97\u5230\u8be5\u5bf9\u8c61\u3002\u5982\u679c\u6ca1\u6709\u7684\u8bdd\uff0c\u8fd4\u56de","None",":","return",":"]
code: def _get_collator(self):
      """
      如果 collate_fn 是 Collator 对象，得到该对象。如果没有的话，返回 None
  
      :return:
      """
      collator = None
      if hasattr(self.collate_fn, "__wrapped__") and isinstance(
          self.collate_fn.__wrapped__, Collator
      ):
          collator = self.collate_fn.__wrapped__
      elif isinstance(self.collate_fn, Collator):
          collator = self.collate_fn
      return collator
  
imports: from typing import Callable, List, Optional, Union, Dict, Sequence
  from copy import deepcopy
  from fastNLP.envs.imports import _NEED_IMPORT_PADDLE
  from fastNLP.core.collators.collator import Collator
  from fastNLP.core.dataloaders.utils import indice_collate_wrapper
  from fastNLP.core.dataset import DataSet
  from fastNLP.core.samplers import ReproducibleBatchSampler, RandomBatchSampler
  from ..utils import _match_param, HasLenGetitemType
label_desc: Use image node_id 4 for calling the PaddleDataLoader obj's underlying member method code with example usage: obj._get_collator() and returns: collator