class_docstr: 用于 pad 数据的对象。会自动将所有能够 pad （由 fastNLP 根据数据判定能否 pad ）的数据都进行 pad 操作，默认 pad 的值为 0。
  判定一个 field 是否可以 pad 的方式为：
  
      1. 当前这个 field 是否所有对象都是一样的数据类型；比如，如果某 field 的数据有些是 float ，有些是 int ，则该 field 将被
         判定为不可 pad 类型；
      2. 当前这个 field 是否每个 sample 都具有一样的深度；比如，如果某 field 的数据转为 batch 类型后为 ``[1, [1,2]]``, 则会
         被判定为不可 pad ，因为第一个 sample 与 第二个 sample 深度不同；
      3. 当前这个 field 的类型是否是可以 pad （例如 str 类型的数据）。可以通过设置 ``logger.setLevel('debug')`` 来打印是判定不可
         pad 的原因。
  
  .. note::
  
      ``Collator`` 的原理是使用第一个 ``batch`` 的数据尝试推断每个 ``field`` 应该使用哪种类型的 ``Padder``，如果第一个 ``batch``
      的数据刚好比较特殊，可能导致在之后的 pad 中遭遇失败，这种情况请通过 :meth:`set_pad` 函数手动设置一下。
  
  .. todo:: 
  
      补充 code example 。
  
  如果需要将某个本可以 pad 的 field 设置为不可 pad ，则可以通过 :meth:`~fastNLP.Collator.set_pad` 的 ``pad_val`` 设置为 ``None`` 实现。
  如果需要某些 field 不要包含在 pad 之后的结果中，可以使用 :meth:`~fastNLP.Collator.set_ignore` 进行设置。
  
  Collator 在第一次进行 pad 的时候自动根据设置以及数据情况，为每个 field 获取一个 padder ，在之后的每次调用中，都将使用对应
  的 Padder 给对应的 field 。由于 Collator 只能在某个 field 内进行 pad ，如果 pad 操作需要同时操作多个 field ，请不要使用 Collator 。
  
  :param backend: 对于可以 pad 的 field，使用哪种 tensor，支持 ``['torch','jittor','paddle','oneflow','numpy','raw', 'auto', None]``。
      若为 ``'auto'`` ，则在进行 pad 的时候会根据调用的环境决定其 ``backend`` 。该参数对不能进行 pad 的数据没有影响，无法 pad 的数据返回一定
      是 :class:`list` 。
class_docstr_tok: ["\u7528\u4e8e","pad","\u6570\u636e\u7684\u5bf9\u8c61\u3002\u4f1a\u81ea\u52a8\u5c06\u6240\u6709\u80fd\u591f","pad","\uff08\u7531","fastNLP","\u6839\u636e\u6570\u636e\u5224\u5b9a\u80fd\u5426","pad","\uff09\u7684\u6570\u636e\u90fd\u8fdb\u884c","pad","\u64cd\u4f5c\uff0c\u9ed8\u8ba4","pad","\u7684\u503c\u4e3a","0\u3002","\u5224\u5b9a\u4e00\u4e2a","field","\u662f\u5426\u53ef\u4ee5","pad","\u7684\u65b9\u5f0f\u4e3a\uff1a","1",".","\u5f53\u524d\u8fd9\u4e2a","field","\u662f\u5426\u6240\u6709\u5bf9\u8c61\u90fd\u662f\u4e00\u6837\u7684\u6570\u636e\u7c7b\u578b\uff1b\u6bd4\u5982\uff0c\u5982\u679c\u67d0","field","\u7684\u6570\u636e\u6709\u4e9b\u662f","float","\uff0c\u6709\u4e9b\u662f","int","\uff0c\u5219\u8be5","field","\u5c06\u88ab","\u5224\u5b9a\u4e3a\u4e0d\u53ef","pad","\u7c7b\u578b\uff1b","2",".","\u5f53\u524d\u8fd9\u4e2a","field","\u662f\u5426\u6bcf\u4e2a","sample","\u90fd\u5177\u6709\u4e00\u6837\u7684\u6df1\u5ea6\uff1b\u6bd4\u5982\uff0c\u5982\u679c\u67d0","field","\u7684\u6570\u636e\u8f6c\u4e3a","batch","\u7c7b\u578b\u540e\u4e3a","``","[","1",",","[","1,2","]","]","``",",","\u5219\u4f1a","\u88ab\u5224\u5b9a\u4e3a\u4e0d\u53ef","pad","\uff0c\u56e0\u4e3a\u7b2c\u4e00\u4e2a","sample","\u4e0e","\u7b2c\u4e8c\u4e2a","sample","\u6df1\u5ea6\u4e0d\u540c\uff1b","3",".","\u5f53\u524d\u8fd9\u4e2a","field","\u7684\u7c7b\u578b\u662f\u5426\u662f\u53ef\u4ee5","pad","\uff08\u4f8b\u5982","str","\u7c7b\u578b\u7684\u6570\u636e\uff09\u3002\u53ef\u4ee5\u901a\u8fc7\u8bbe\u7f6e","``","logger.setLevel","(","'debug","'",")","``","\u6765\u6253\u5370\u662f\u5224\u5b9a\u4e0d\u53ef","pad","\u7684\u539f\u56e0\u3002","..","note",":",":","``","Collator","``","\u7684\u539f\u7406\u662f\u4f7f\u7528\u7b2c\u4e00\u4e2a","``","batch","``","\u7684\u6570\u636e\u5c1d\u8bd5\u63a8\u65ad\u6bcf\u4e2a","``","field","``","\u5e94\u8be5\u4f7f\u7528\u54ea\u79cd\u7c7b\u578b\u7684","``","Padder","``","\uff0c\u5982\u679c\u7b2c\u4e00\u4e2a","``","batch","``","\u7684\u6570\u636e\u521a\u597d\u6bd4\u8f83\u7279\u6b8a\uff0c\u53ef\u80fd\u5bfc\u81f4\u5728\u4e4b\u540e\u7684","pad","\u4e2d\u906d\u9047\u5931\u8d25\uff0c\u8fd9\u79cd\u60c5\u51b5\u8bf7\u901a\u8fc7",":","meth",":","`","set_pad","`","\u51fd\u6570\u624b\u52a8\u8bbe\u7f6e\u4e00\u4e0b\u3002","..","todo",":",":","\u8865\u5145","code","example","\u3002","\u5982\u679c\u9700\u8981\u5c06\u67d0\u4e2a\u672c\u53ef\u4ee5","pad","\u7684","field","\u8bbe\u7f6e\u4e3a\u4e0d\u53ef","pad","\uff0c\u5219\u53ef\u4ee5\u901a\u8fc7",":","meth",":","`","~fastNLP.Collator.set_pad","`","\u7684","``","pad_val","``","\u8bbe\u7f6e\u4e3a","``","None","``","\u5b9e\u73b0\u3002","\u5982\u679c\u9700\u8981\u67d0\u4e9b","field","\u4e0d\u8981\u5305\u542b\u5728","pad","\u4e4b\u540e\u7684\u7ed3\u679c\u4e2d\uff0c\u53ef\u4ee5\u4f7f\u7528",":","meth",":","`","~fastNLP.Collator.set_ignore","`","\u8fdb\u884c\u8bbe\u7f6e\u3002","Collator","\u5728\u7b2c\u4e00\u6b21\u8fdb\u884c","pad","\u7684\u65f6\u5019\u81ea\u52a8\u6839\u636e\u8bbe\u7f6e\u4ee5\u53ca\u6570\u636e\u60c5\u51b5\uff0c\u4e3a\u6bcf\u4e2a","field","\u83b7\u53d6\u4e00\u4e2a","padder","\uff0c\u5728\u4e4b\u540e\u7684\u6bcf\u6b21\u8c03\u7528\u4e2d\uff0c\u90fd\u5c06\u4f7f\u7528\u5bf9\u5e94","\u7684","Padder","\u7ed9\u5bf9\u5e94\u7684","field","\u3002\u7531\u4e8e","Collator","\u53ea\u80fd\u5728\u67d0\u4e2a","field","\u5185\u8fdb\u884c","pad","\uff0c\u5982\u679c","pad","\u64cd\u4f5c\u9700\u8981\u540c\u65f6\u64cd\u4f5c\u591a\u4e2a","field","\uff0c\u8bf7\u4e0d\u8981\u4f7f\u7528","Collator","\u3002",":","param","backend",":","\u5bf9\u4e8e\u53ef\u4ee5","pad","\u7684","field\uff0c\u4f7f\u7528\u54ea\u79cd","tensor\uff0c\u652f\u6301","``","[","'torch","'",",","'jittor","'",",","'paddle","'",",","'oneflow","'",",","'numpy","'",",","'raw","'",",","'auto","'",",","None","]","``","\u3002","\u82e5\u4e3a","``","'auto","'","``","\uff0c\u5219\u5728\u8fdb\u884c","pad","\u7684\u65f6\u5019\u4f1a\u6839\u636e\u8c03\u7528\u7684\u73af\u5883\u51b3\u5b9a\u5176","``","backend","``","\u3002\u8be5\u53c2\u6570\u5bf9\u4e0d\u80fd\u8fdb\u884c","pad","\u7684\u6570\u636e\u6ca1\u6709\u5f71\u54cd\uff0c\u65e0\u6cd5","pad","\u7684\u6570\u636e\u8fd4\u56de\u4e00\u5b9a","\u662f",":","class",":","`","list","`","\u3002"]
docstr: batch可能存在三种可能性：List[Dict], List[List], List[Sample]
  
  第一步：使用 unpack_batch_func 将相同 field 的内容打包到一个 list 中。
  第二步：使用每个 field 各自的 padder 进行 pad 。
  第三步：根据 batch 中每个 sample 的类型，返回也保证为该类型。
  
  第一次调用会根据当前 batch 数据决定使用哪个 unpack_batch_func ，这个函数的作用是把不同 sample 的同一个 field 的放入到一个
      list 中；同时也会决定 pack_batch_func，这个函数的作用是在返回 pad 好的 batch 之前，将 batch 恢复为 输入时一个 sample
      的类别。
  第一次调用会根据当前 field 决定对应的 Padder 。
docstr_tok: ["batch\u53ef\u80fd\u5b58\u5728\u4e09\u79cd\u53ef\u80fd\u6027\uff1aList","[","Dict","]",",","List","[","List","]",",","List","[","Sample","]","\u7b2c\u4e00\u6b65\uff1a\u4f7f\u7528","unpack_batch_func","\u5c06\u76f8\u540c","field","\u7684\u5185\u5bb9\u6253\u5305\u5230\u4e00\u4e2a","list","\u4e2d\u3002","\u7b2c\u4e8c\u6b65\uff1a\u4f7f\u7528\u6bcf\u4e2a","field","\u5404\u81ea\u7684","padder","\u8fdb\u884c","pad","\u3002","\u7b2c\u4e09\u6b65\uff1a\u6839\u636e","batch","\u4e2d\u6bcf\u4e2a","sample","\u7684\u7c7b\u578b\uff0c\u8fd4\u56de\u4e5f\u4fdd\u8bc1\u4e3a\u8be5\u7c7b\u578b\u3002","\u7b2c\u4e00\u6b21\u8c03\u7528\u4f1a\u6839\u636e\u5f53\u524d","batch","\u6570\u636e\u51b3\u5b9a\u4f7f\u7528\u54ea\u4e2a","unpack_batch_func","\uff0c\u8fd9\u4e2a\u51fd\u6570\u7684\u4f5c\u7528\u662f\u628a\u4e0d\u540c","sample","\u7684\u540c\u4e00\u4e2a","field","\u7684\u653e\u5165\u5230\u4e00\u4e2a","list","\u4e2d\uff1b\u540c\u65f6\u4e5f\u4f1a\u51b3\u5b9a","pack_batch_func\uff0c\u8fd9\u4e2a\u51fd\u6570\u7684\u4f5c\u7528\u662f\u5728\u8fd4\u56de","pad","\u597d\u7684","batch","\u4e4b\u524d\uff0c\u5c06","batch","\u6062\u590d\u4e3a","\u8f93\u5165\u65f6\u4e00\u4e2a","sample","\u7684\u7c7b\u522b\u3002","\u7b2c\u4e00\u6b21\u8c03\u7528\u4f1a\u6839\u636e\u5f53\u524d","field","\u51b3\u5b9a\u5bf9\u5e94\u7684","Padder","\u3002"]
code: def __call__(self, batch) -> Union[List, Dict]:
      """
      batch可能存在三种可能性：List[Dict], List[List], List[Sample]
  
      第一步：使用 unpack_batch_func 将相同 field 的内容打包到一个 list 中。
      第二步：使用每个 field 各自的 padder 进行 pad 。
      第三步：根据 batch 中每个 sample 的类型，返回也保证为该类型。
  
      第一次调用会根据当前 batch 数据决定使用哪个 unpack_batch_func ，这个函数的作用是把不同 sample 的同一个 field 的放入到一个
          list 中；同时也会决定 pack_batch_func，这个函数的作用是在返回 pad 好的 batch 之前，将 batch 恢复为 输入时一个 sample
          的类别。
      第一次调用会根据当前 field 决定对应的 Padder 。
  
      """
      if self.unpack_batch_func is None:
          # 决定使用哪个unpack_batch_func，让它都 return 回 dict 类型
          if self.batch_data_type is None:
              if isinstance(batch[0], Mapping):
                  self.batch_data_type = "d"
              elif isinstance(batch[0], Sequence):  # 这里存在误判的风险
                  self.batch_data_type = "l"
              else:
                  self.batch_data_type = "s"
              logger.debug(
                  f"Since batch[0] has type:{type(batch[0])}, so the batch_data_type "
                  f"is `{self.batch_data_type}`."
              )
          if self.batch_data_type == "s":
              self.packer_unpacker = SinglePackerUnpacker()  # 不需要做任何调整
          elif self.batch_data_type == "l":
              self.packer_unpacker = SequencePackerUnpacker()
          elif self.batch_data_type == "d":
              if any(
                  [isinstance(v, Mapping) for v in batch[0].values()]
              ):  # 可能存在 nested 的dict。{'a': {'b': xx}}->{('a', 'b'): value}
                  self.packer_unpacker = NestedMappingPackerUnpacker()
              else:
                  self.packer_unpacker = MappingPackerUnpacker()
  
      # 将 batch 中各个 field 组成自己的 batch；同时忽略处于 ignore_fields 中的数据。
      unpack_batch = self.packer_unpacker.unpack_batch(
          batch, self.ignore_fields, self.input_fields
      )
  
      pad_batch = {}
      if len(self.padders) == 0:  # 第一次运行，准备 padder
          if (
              self.backend == "auto"
          ):  # 如果 backend 为 auto ，则尝试通过调用栈等自动获取 backend 。
              self.backend = _get_backend()
  
          for field_name, batch_field in unpack_batch.items():
              setting = self.input_fields.get(
                  field_name,
                  {
                      "backend": self.backend,
                      "pad_val": 0,
                      "dtype": None,
                      "pad_fn": None,
                  },
              )
              pad_fn = setting["pad_fn"]
              if callable(pad_fn):
                  padder = pad_fn
              else:
                  backend = (
                      self.backend
                      if setting["backend"] == "auto"
                      else setting["backend"]
                  )
                  padder = get_padder(
                      batch_field=batch_field,
                      pad_val=setting["pad_val"],
                      dtype=setting["dtype"],
                      backend=backend,
                      field_name=field_name,
                  )
              self.padders[field_name] = padder
  
          if self.batch_data_type == "l":
              self.padders = dict(
                  sorted(
                      self.padders.items(), key=lambda x: int(x[0][1:])
                  )
              )  # sort, 这样 _0, _1 能够保持顺序
      try:
          for key, padder in self.padders.items():
              batch = unpack_batch.get(key)
              pad_batch[key] = padder(batch)
      except BaseException as e:
          try:
              logger.error(
                  f"The following exception happens when try to pad the `{key}` field with padder:{padder}:"
              )
          except:
              pass
          raise e
  
      return self.packer_unpacker.pack_batch(pad_batch)
  
file_path: fastNLP/fastNLP/core/collators/collator.py
imports: from typing import List, Union, Dict, Callable, Sequence, Mapping
  import os
  import sys
  import inspect
  import re
  from fastNLP.core.log import logger
  from .padders.get_padder import get_padder
  from ...envs import SUPPORT_BACKENDS
  from .padders import Padder
  from .packer_unpacker import SequencePackerUnpacker, SinglePackerUnpacker, MappingPackerUnpacker, NestedMappingPackerUnpacker
label_desc: Use image node_id 2 for calling the Collator obj's underlying member method code with example usage: obj.__call__(batch) and returns: self