class_docstr: 用来初始化 `Driver` 的基类，所有定制的 `driver` 都需要继承此类；
  **fastNLP** 提供的 driver 实例都会同时被 :class:`~fastNLP.core.controllers.Trainer` 和 :class:`~fastNLP.core.controllers.Evaluator` 调用。
  
  :param model: 训练或者评测的模型，需要注意该模型可能为用户已经使用类似 :class:`torch.nn.DataParallel` 或者
      :class:`torch.nn.parallel.DistributedDataParallel` 包裹过的模型。
class_docstr_tok: ["\u7528\u6765\u521d\u59cb\u5316","`","Driver","`","\u7684\u57fa\u7c7b\uff0c\u6240\u6709\u5b9a\u5236\u7684","`","driver","`","\u90fd\u9700\u8981\u7ee7\u627f\u6b64\u7c7b\uff1b","*","*","fastNLP","*","*","\u63d0\u4f9b\u7684","driver","\u5b9e\u4f8b\u90fd\u4f1a\u540c\u65f6\u88ab",":","class",":","`","~fastNLP.core.controllers.Trainer","`","\u548c",":","class",":","`","~fastNLP.core.controllers.Evaluator","`","\u8c03\u7528\u3002",":","param","model",":","\u8bad\u7ec3\u6216\u8005\u8bc4\u6d4b\u7684\u6a21\u578b\uff0c\u9700\u8981\u6ce8\u610f\u8be5\u6a21\u578b\u53ef\u80fd\u4e3a\u7528\u6237\u5df2\u7ecf\u4f7f\u7528\u7c7b\u4f3c",":","class",":","`","torch.nn.DataParallel","`","\u6216\u8005",":","class",":","`","torch.nn.parallel.DistributedDataParallel","`","\u5305\u88f9\u8fc7\u7684\u6a21\u578b\u3002"]
docstr: 断点重训的加载函数，该函数会负责读取数据，并且恢复优化器 、sampler 的状态和模型（如果 ``should_load_model`` 为 True）以及其它在 :meth:`save_checkpoint`
  函数中执行的保存操作，然后将一个 state 字典返回给 :class:`~fastNLP.core.controllers.Trainer` （ 内容为 :meth:`save_checkpoint` 接受到的 ``states`` ）。
  
  该函数应该在所有 rank 上执行。
  
  :param folder: 读取该 folder 下的 ``FASTNLP_CHECKPOINT_FILENAME`` 文件与 ``FASTNLP_MODEL_FILENAME``
      （如果 should_load_model 为True）。
  :param dataloader: 当前给定 dataloader，需要根据保存的 dataloader 状态合理设置。若该值为 ``None`` ，则不需要返回 ``'dataloader'``
      以及 ``'batch_idx_in_epoch'`` 这两个值。
  :param only_state_dict: 是否仅读取模型的 state_dict ，当 ``should_save_model`` 为 ``False`` ，该参数无效。如果为 ``True`` ，说明保存的内容为权重；如果为
      False 说明保存的是模型，但也是通过当前 Driver 的模型去加载保存的模型的权重，而不是使用保存的模型替换当前模型。
  :param should_load_model: 是否应该加载模型，如果为 ``False`` ，Driver 将不负责加载模型。若该参数为 ``True`` ，但在保存的状态中没有
      找到对应的模型状态，则报错。
  :return: :meth:`save_checkpoint` 函数输入的 ``states`` 内容。除此之外，还返回的内容有：
  
      * *dataloader* -- 根据传入的 ``dataloader`` 与读取出的状态设置为合理状态的 dataloader。在当前 ``dataloader`` 样本数与读取出的 sampler 样本数
        不一致时报错。
      * *batch_idx_in_epoch* -- :class:`int` 类型的数据，表明当前 epoch 进行到了第几个 batch 。请注意，该值不能仅通过保存的数据中读取的，因为前后两次运行的
        ``batch_size`` 可能有变化，而应该符合以下等式::
  
          返回的 dataloader 还会产生的 batch 数量 + batch_idx_in_epoch = 原来不断点训练时的 batch 的总数
        
        由于 ``返回的 dataloader 还会产生的batch数`` 在 ``batch_size`` 与 ``drop_last`` 参数给定的情况下，无法改变，因此只能通过调整 ``batch_idx_in_epoch``
        这个值来使等式成立。一个简单的计算原则如下：
  
          * drop_last 为 ``True`` 时，等同于 floor(sample_in_this_rank/batch_size) - floor(num_left_samples/batch_size)；
          * drop_last 为 ``False`` 时，等同于 ceil(sample_in_this_rank/batch_size) - ceil(num_left_samples/batch_size)。
docstr_tok: ["\u65ad\u70b9\u91cd\u8bad\u7684\u52a0\u8f7d\u51fd\u6570\uff0c\u8be5\u51fd\u6570\u4f1a\u8d1f\u8d23\u8bfb\u53d6\u6570\u636e\uff0c\u5e76\u4e14\u6062\u590d\u4f18\u5316\u5668","\u3001sampler","\u7684\u72b6\u6001\u548c\u6a21\u578b\uff08\u5982\u679c","``","should_load_model","``","\u4e3a","True\uff09\u4ee5\u53ca\u5176\u5b83\u5728",":","meth",":","`","save_checkpoint","`","\u51fd\u6570\u4e2d\u6267\u884c\u7684\u4fdd\u5b58\u64cd\u4f5c\uff0c\u7136\u540e\u5c06\u4e00\u4e2a","state","\u5b57\u5178\u8fd4\u56de\u7ed9",":","class",":","`","~fastNLP.core.controllers.Trainer","`","\uff08","\u5185\u5bb9\u4e3a",":","meth",":","`","save_checkpoint","`","\u63a5\u53d7\u5230\u7684","``","states","``","\uff09\u3002","\u8be5\u51fd\u6570\u5e94\u8be5\u5728\u6240\u6709","rank","\u4e0a\u6267\u884c\u3002",":","param","folder",":","\u8bfb\u53d6\u8be5","folder","\u4e0b\u7684","``","FASTNLP_CHECKPOINT_FILENAME","``","\u6587\u4ef6\u4e0e","``","FASTNLP_MODEL_FILENAME","``","\uff08\u5982\u679c","should_load_model","\u4e3aTrue\uff09\u3002",":","param","dataloader",":","\u5f53\u524d\u7ed9\u5b9a","dataloader\uff0c\u9700\u8981\u6839\u636e\u4fdd\u5b58\u7684","dataloader","\u72b6\u6001\u5408\u7406\u8bbe\u7f6e\u3002\u82e5\u8be5\u503c\u4e3a","``","None","``","\uff0c\u5219\u4e0d\u9700\u8981\u8fd4\u56de","``","'dataloader","'","``","\u4ee5\u53ca","``","'batch_idx_in_epoch","'","``","\u8fd9\u4e24\u4e2a\u503c\u3002",":","param","only_state_dict",":","\u662f\u5426\u4ec5\u8bfb\u53d6\u6a21\u578b\u7684","state_dict","\uff0c\u5f53","``","should_save_model","``","\u4e3a","``","False","``","\uff0c\u8be5\u53c2\u6570\u65e0\u6548\u3002\u5982\u679c\u4e3a","``","True","``","\uff0c\u8bf4\u660e\u4fdd\u5b58\u7684\u5185\u5bb9\u4e3a\u6743\u91cd\uff1b\u5982\u679c\u4e3a","False","\u8bf4\u660e\u4fdd\u5b58\u7684\u662f\u6a21\u578b\uff0c\u4f46\u4e5f\u662f\u901a\u8fc7\u5f53\u524d","Driver","\u7684\u6a21\u578b\u53bb\u52a0\u8f7d\u4fdd\u5b58\u7684\u6a21\u578b\u7684\u6743\u91cd\uff0c\u800c\u4e0d\u662f\u4f7f\u7528\u4fdd\u5b58\u7684\u6a21\u578b\u66ff\u6362\u5f53\u524d\u6a21\u578b\u3002",":","param","should_load_model",":","\u662f\u5426\u5e94\u8be5\u52a0\u8f7d\u6a21\u578b\uff0c\u5982\u679c\u4e3a","``","False","``","\uff0cDriver","\u5c06\u4e0d\u8d1f\u8d23\u52a0\u8f7d\u6a21\u578b\u3002\u82e5\u8be5\u53c2\u6570\u4e3a","``","True","``","\uff0c\u4f46\u5728\u4fdd\u5b58\u7684\u72b6\u6001\u4e2d\u6ca1\u6709","\u627e\u5230\u5bf9\u5e94\u7684\u6a21\u578b\u72b6\u6001\uff0c\u5219\u62a5\u9519\u3002",":","return",":",":","meth",":","`","save_checkpoint","`","\u51fd\u6570\u8f93\u5165\u7684","``","states","``","\u5185\u5bb9\u3002\u9664\u6b64\u4e4b\u5916\uff0c\u8fd8\u8fd4\u56de\u7684\u5185\u5bb9\u6709\uff1a","*","*","dataloader","*","--","\u6839\u636e\u4f20\u5165\u7684","``","dataloader","``","\u4e0e\u8bfb\u53d6\u51fa\u7684\u72b6\u6001\u8bbe\u7f6e\u4e3a\u5408\u7406\u72b6\u6001\u7684","dataloader\u3002\u5728\u5f53\u524d","``","dataloader","``","\u6837\u672c\u6570\u4e0e\u8bfb\u53d6\u51fa\u7684","sampler","\u6837\u672c\u6570","\u4e0d\u4e00\u81f4\u65f6\u62a5\u9519\u3002","*","*","batch_idx_in_epoch","*","--",":","class",":","`","int","`","\u7c7b\u578b\u7684\u6570\u636e\uff0c\u8868\u660e\u5f53\u524d","epoch","\u8fdb\u884c\u5230\u4e86\u7b2c\u51e0\u4e2a","batch","\u3002\u8bf7\u6ce8\u610f\uff0c\u8be5\u503c\u4e0d\u80fd\u4ec5\u901a\u8fc7\u4fdd\u5b58\u7684\u6570\u636e\u4e2d\u8bfb\u53d6\u7684\uff0c\u56e0\u4e3a\u524d\u540e\u4e24\u6b21\u8fd0\u884c\u7684","``","batch_size","``","\u53ef\u80fd\u6709\u53d8\u5316\uff0c\u800c\u5e94\u8be5\u7b26\u5408\u4ee5\u4e0b\u7b49\u5f0f",":",":","\u8fd4\u56de\u7684","dataloader","\u8fd8\u4f1a\u4ea7\u751f\u7684","batch","\u6570\u91cf","+","batch_idx_in_epoch","=","\u539f\u6765\u4e0d\u65ad\u70b9\u8bad\u7ec3\u65f6\u7684","batch","\u7684\u603b\u6570","\u7531\u4e8e","``","\u8fd4\u56de\u7684","dataloader","\u8fd8\u4f1a\u4ea7\u751f\u7684batch\u6570","``","\u5728","``","batch_size","``","\u4e0e","``","drop_last","``","\u53c2\u6570\u7ed9\u5b9a\u7684\u60c5\u51b5\u4e0b\uff0c\u65e0\u6cd5\u6539\u53d8\uff0c\u56e0\u6b64\u53ea\u80fd\u901a\u8fc7\u8c03\u6574","``","batch_idx_in_epoch","``","\u8fd9\u4e2a\u503c\u6765\u4f7f\u7b49\u5f0f\u6210\u7acb\u3002\u4e00\u4e2a\u7b80\u5355\u7684\u8ba1\u7b97\u539f\u5219\u5982\u4e0b\uff1a","*","drop_last","\u4e3a","``","True","``","\u65f6\uff0c\u7b49\u540c\u4e8e","floor","(","sample_in_this_rank\/batch_size",")","-","floor","(","num_left_samples\/batch_size",")","\uff1b","*","drop_last","\u4e3a","``","False","``","\u65f6\uff0c\u7b49\u540c\u4e8e","ceil","(","sample_in_this_rank\/batch_size",")","-","ceil","(","num_left_samples\/batch_size",")","\u3002"]
code: def load_checkpoint(
      self,
      folder: Union[str, Path],
      dataloader,
      only_state_dict: bool = True,
      should_load_model: bool = True,
      **kwargs,
  ) -> Dict:
      r"""
      断点重训的加载函数，该函数会负责读取数据，并且恢复优化器 、sampler 的状态和模型（如果 ``should_load_model`` 为 True）以及其它在 :meth:`save_checkpoint`
      函数中执行的保存操作，然后将一个 state 字典返回给 :class:`~fastNLP.core.controllers.Trainer` （ 内容为 :meth:`save_checkpoint` 接受到的 ``states`` ）。
  
      该函数应该在所有 rank 上执行。
  
      :param folder: 读取该 folder 下的 ``FASTNLP_CHECKPOINT_FILENAME`` 文件与 ``FASTNLP_MODEL_FILENAME``
          （如果 should_load_model 为True）。
      :param dataloader: 当前给定 dataloader，需要根据保存的 dataloader 状态合理设置。若该值为 ``None`` ，则不需要返回 ``'dataloader'``
          以及 ``'batch_idx_in_epoch'`` 这两个值。
      :param only_state_dict: 是否仅读取模型的 state_dict ，当 ``should_save_model`` 为 ``False`` ，该参数无效。如果为 ``True`` ，说明保存的内容为权重；如果为
          False 说明保存的是模型，但也是通过当前 Driver 的模型去加载保存的模型的权重，而不是使用保存的模型替换当前模型。
      :param should_load_model: 是否应该加载模型，如果为 ``False`` ，Driver 将不负责加载模型。若该参数为 ``True`` ，但在保存的状态中没有
          找到对应的模型状态，则报错。
      :return: :meth:`save_checkpoint` 函数输入的 ``states`` 内容。除此之外，还返回的内容有：
  
          * *dataloader* -- 根据传入的 ``dataloader`` 与读取出的状态设置为合理状态的 dataloader。在当前 ``dataloader`` 样本数与读取出的 sampler 样本数
            不一致时报错。
          * *batch_idx_in_epoch* -- :class:`int` 类型的数据，表明当前 epoch 进行到了第几个 batch 。请注意，该值不能仅通过保存的数据中读取的，因为前后两次运行的
            ``batch_size`` 可能有变化，而应该符合以下等式::
  
              返回的 dataloader 还会产生的 batch 数量 + batch_idx_in_epoch = 原来不断点训练时的 batch 的总数
  
            由于 ``返回的 dataloader 还会产生的batch数`` 在 ``batch_size`` 与 ``drop_last`` 参数给定的情况下，无法改变，因此只能通过调整 ``batch_idx_in_epoch``
            这个值来使等式成立。一个简单的计算原则如下：
  
              * drop_last 为 ``True`` 时，等同于 floor(sample_in_this_rank/batch_size) - floor(num_left_samples/batch_size)；
              * drop_last 为 ``False`` 时，等同于 ceil(sample_in_this_rank/batch_size) - ceil(num_left_samples/batch_size)。
      """
  
imports: import os
  import signal
  import sys
  from typing import Sequence, List, Optional, Callable, Dict, Union, Tuple
  from abc import ABC, abstractmethod
  from datetime import datetime
  from pathlib import Path
  from io import BytesIO
  import json
  from fastNLP.core.utils import nullcontext
label_desc: Use image node_id 28 for calling the Driver obj's underlying member method code with example usage: obj.load_checkpoint(folder, dataloader, only_state_dict, should_load_model) without return types