class_docstr: 各种数据 **Loader** 的基类，提供了 API 的参考。
  :class:`Loader` 支持以下的三个函数
  
  - :meth:`download` 函数：自动将该数据集下载到缓存地址，默认缓存地址为 ``~/.fastNLP/datasets/`` 。由于版权等原因，不是所有的 ``Loader`` 都实现了该方法。
    该方法会返回下载后文件所处的缓存地址。
  - :meth:`_load` 函数：从一个数据文件中读取数据，返回一个 :class:`~fastNLP.core.DataSet` 。返回的 DataSet 的内容可以通过每个 ``Loader`` 的文档判断出。
  - :meth:`load` 函数：将文件分别读取为 :class:`~fastNLP.core.DataSet` ，然后将多个 DataSet 放入到一个 :class:`~fastNLP.io.DataBundle` 中并返回
class_docstr_tok: ["\u5404\u79cd\u6570\u636e","*","*","Loader","*","*","\u7684\u57fa\u7c7b\uff0c\u63d0\u4f9b\u4e86","API","\u7684\u53c2\u8003\u3002",":","class",":","`","Loader","`","\u652f\u6301\u4ee5\u4e0b\u7684\u4e09\u4e2a\u51fd\u6570","-",":","meth",":","`","download","`","\u51fd\u6570\uff1a\u81ea\u52a8\u5c06\u8be5\u6570\u636e\u96c6\u4e0b\u8f7d\u5230\u7f13\u5b58\u5730\u5740\uff0c\u9ed8\u8ba4\u7f13\u5b58\u5730\u5740\u4e3a","``","~\/.fastNLP\/datasets\/","``","\u3002\u7531\u4e8e\u7248\u6743\u7b49\u539f\u56e0\uff0c\u4e0d\u662f\u6240\u6709\u7684","``","Loader","``","\u90fd\u5b9e\u73b0\u4e86\u8be5\u65b9\u6cd5\u3002","\u8be5\u65b9\u6cd5\u4f1a\u8fd4\u56de\u4e0b\u8f7d\u540e\u6587\u4ef6\u6240\u5904\u7684\u7f13\u5b58\u5730\u5740\u3002","-",":","meth",":","`","_load","`","\u51fd\u6570\uff1a\u4ece\u4e00\u4e2a\u6570\u636e\u6587\u4ef6\u4e2d\u8bfb\u53d6\u6570\u636e\uff0c\u8fd4\u56de\u4e00\u4e2a",":","class",":","`","~fastNLP.core.DataSet","`","\u3002\u8fd4\u56de\u7684","DataSet","\u7684\u5185\u5bb9\u53ef\u4ee5\u901a\u8fc7\u6bcf\u4e2a","``","Loader","``","\u7684\u6587\u6863\u5224\u65ad\u51fa\u3002","-",":","meth",":","`","load","`","\u51fd\u6570\uff1a\u5c06\u6587\u4ef6\u5206\u522b\u8bfb\u53d6\u4e3a",":","class",":","`","~fastNLP.core.DataSet","`","\uff0c\u7136\u540e\u5c06\u591a\u4e2a","DataSet","\u653e\u5165\u5230\u4e00\u4e2a",":","class",":","`","~fastNLP.io.DataBundle","`","\u4e2d\u5e76\u8fd4\u56de"]
docstr: 从指定一个或多个路径中的文件中读取数据，返回 :class:`~fastNLP.io.DataBundle` 。
  
  :param paths: 支持以下的几种输入方式：
  
      - ``None`` -- 先查看本地是否有缓存，如果没有则自动下载并缓存。
      - 一个目录，该目录下名称包含 ``'train'`` 的被认为是训练集，包含 ``'test'`` 的被认为是测试集，包含 ``'dev'`` 的被认为是验证集 / 开发集，
        如果检测到多个文件名包含 ``'train'``、 ``'dev'``、 ``'test'`` 则会报错::
  
          data_bundle = xxxLoader().load('/path/to/dir')  # 返回的DataBundle中datasets根据目录下是否检测到train
          #  dev、 test 等有所变化，可以通过以下的方式取出 DataSet
          tr_data = data_bundle.get_dataset('train')
          te_data = data_bundle.get_dataset('test')  # 如果目录下有文件包含test这个字段
  
      - 传入一个 :class:`dict` ，比如训练集、验证集和测试集不在同一个目录下，或者名称中不包含 ``'train'``、 ``'dev'``、 ``'test'`` ::
  
          paths = {'train':"/path/to/tr.conll", 'dev':"/to/validate.conll", "test":"/to/te.conll"}
          data_bundle = xxxLoader().load(paths)  # 返回的DataBundle中的dataset中包含"train", "dev", "test"
          dev_data = data_bundle.get_dataset('dev')
  
      - 传入文件路径::
  
          data_bundle = xxxLoader().load("/path/to/a/train.conll") # 返回DataBundle对象, datasets中仅包含'train'
          tr_data = data_bundle.get_dataset('train')  # 取出DataSet
  
  :return: :class:`~fastNLP.io.DataBundle`
docstr_tok: ["\u4ece\u6307\u5b9a\u4e00\u4e2a\u6216\u591a\u4e2a\u8def\u5f84\u4e2d\u7684\u6587\u4ef6\u4e2d\u8bfb\u53d6\u6570\u636e\uff0c\u8fd4\u56de",":","class",":","`","~fastNLP.io.DataBundle","`","\u3002",":","param","paths",":","\u652f\u6301\u4ee5\u4e0b\u7684\u51e0\u79cd\u8f93\u5165\u65b9\u5f0f\uff1a","-","``","None","``","--","\u5148\u67e5\u770b\u672c\u5730\u662f\u5426\u6709\u7f13\u5b58\uff0c\u5982\u679c\u6ca1\u6709\u5219\u81ea\u52a8\u4e0b\u8f7d\u5e76\u7f13\u5b58\u3002","-","\u4e00\u4e2a\u76ee\u5f55\uff0c\u8be5\u76ee\u5f55\u4e0b\u540d\u79f0\u5305\u542b","``","'train","'","``","\u7684\u88ab\u8ba4\u4e3a\u662f\u8bad\u7ec3\u96c6\uff0c\u5305\u542b","``","'test","'","``","\u7684\u88ab\u8ba4\u4e3a\u662f\u6d4b\u8bd5\u96c6\uff0c\u5305\u542b","``","'dev","'","``","\u7684\u88ab\u8ba4\u4e3a\u662f\u9a8c\u8bc1\u96c6","\/","\u5f00\u53d1\u96c6\uff0c","\u5982\u679c\u68c0\u6d4b\u5230\u591a\u4e2a\u6587\u4ef6\u540d\u5305\u542b","``","'train","'","``","\u3001","``","'dev","'","``","\u3001","``","'test","'","``","\u5219\u4f1a\u62a5\u9519",":",":","data_bundle","=","xxxLoader","(",")",".load","(","'\/path\/to\/dir","'",")","#","\u8fd4\u56de\u7684DataBundle\u4e2ddatasets\u6839\u636e\u76ee\u5f55\u4e0b\u662f\u5426\u68c0\u6d4b\u5230train","#","dev\u3001","test","\u7b49\u6709\u6240\u53d8\u5316\uff0c\u53ef\u4ee5\u901a\u8fc7\u4ee5\u4e0b\u7684\u65b9\u5f0f\u53d6\u51fa","DataSet","tr_data","=","data_bundle.get_dataset","(","'train","'",")","te_data","=","data_bundle.get_dataset","(","'test","'",")","#","\u5982\u679c\u76ee\u5f55\u4e0b\u6709\u6587\u4ef6\u5305\u542btest\u8fd9\u4e2a\u5b57\u6bb5","-","\u4f20\u5165\u4e00\u4e2a",":","class",":","`","dict","`","\uff0c\u6bd4\u5982\u8bad\u7ec3\u96c6\u3001\u9a8c\u8bc1\u96c6\u548c\u6d4b\u8bd5\u96c6\u4e0d\u5728\u540c\u4e00\u4e2a\u76ee\u5f55\u4e0b\uff0c\u6216\u8005\u540d\u79f0\u4e2d\u4e0d\u5305\u542b","``","'train","'","``","\u3001","``","'dev","'","``","\u3001","``","'test","'","``",":",":","paths","=","{","'train","'",":","''","\/path\/to\/tr.conll","''",",","'dev","'",":","''","\/to\/validate.conll","''",",","``","test","''",":","''","\/to\/te.conll","''","}","data_bundle","=","xxxLoader","(",")",".load","(","paths",")","#","\u8fd4\u56de\u7684DataBundle\u4e2d\u7684dataset\u4e2d\u5305\u542b","''","train","''",",","``","dev","''",",","``","test","''","dev_data","=","data_bundle.get_dataset","(","'dev","'",")","-","\u4f20\u5165\u6587\u4ef6\u8def\u5f84",":",":","data_bundle","=","xxxLoader","(",")",".load","(","``","\/path\/to\/a\/train.conll","''",")","#","\u8fd4\u56deDataBundle\u5bf9\u8c61",",","datasets\u4e2d\u4ec5\u5305\u542b'train'","tr_data","=","data_bundle.get_dataset","(","'train","'",")","#","\u53d6\u51faDataSet",":","return",":",":","class",":","`","~fastNLP.io.DataBundle","`"]
code: def load(
      self, paths: Union[str, Dict[str, str]] = None
  ) -> DataBundle:
      r"""
      从指定一个或多个路径中的文件中读取数据，返回 :class:`~fastNLP.io.DataBundle` 。
  
      :param paths: 支持以下的几种输入方式：
  
          - ``None`` -- 先查看本地是否有缓存，如果没有则自动下载并缓存。
          - 一个目录，该目录下名称包含 ``'train'`` 的被认为是训练集，包含 ``'test'`` 的被认为是测试集，包含 ``'dev'`` 的被认为是验证集 / 开发集，
            如果检测到多个文件名包含 ``'train'``、 ``'dev'``、 ``'test'`` 则会报错::
  
              data_bundle = xxxLoader().load('/path/to/dir')  # 返回的DataBundle中datasets根据目录下是否检测到train
              #  dev、 test 等有所变化，可以通过以下的方式取出 DataSet
              tr_data = data_bundle.get_dataset('train')
              te_data = data_bundle.get_dataset('test')  # 如果目录下有文件包含test这个字段
  
          - 传入一个 :class:`dict` ，比如训练集、验证集和测试集不在同一个目录下，或者名称中不包含 ``'train'``、 ``'dev'``、 ``'test'`` ::
  
              paths = {'train':"/path/to/tr.conll", 'dev':"/to/validate.conll", "test":"/to/te.conll"}
              data_bundle = xxxLoader().load(paths)  # 返回的DataBundle中的dataset中包含"train", "dev", "test"
              dev_data = data_bundle.get_dataset('dev')
  
          - 传入文件路径::
  
              data_bundle = xxxLoader().load("/path/to/a/train.conll") # 返回DataBundle对象, datasets中仅包含'train'
              tr_data = data_bundle.get_dataset('train')  # 取出DataSet
  
      :return: :class:`~fastNLP.io.DataBundle`
      """
      if paths is None:
          paths = self.download()
      paths = check_loader_paths(paths)
      datasets = {
          name: self._load(path) for name, path in paths.items()
      }
      data_bundle = DataBundle(datasets=datasets)
      return data_bundle
  
file_path: fastNLP/fastNLP/io/loader/loader.py
imports: from typing import Union, Dict
  from fastNLP.io.data_bundle import DataBundle
  from fastNLP.io.file_utils import _get_dataset_url, get_cache_path, cached_path
  from fastNLP.io.utils import check_loader_paths
  from fastNLP.core.dataset import DataSet
label_desc: Use image node_id 3 for calling the Loader obj's underlying member method code with example usage: obj.load(paths) and returns: data_bundle