class_docstr: Loads a saved AdaptiveModel/ONNXAdaptiveModel from disk and runs it in inference mode. Can be used for a
  model with prediction head (down-stream predictions) and without (using LM as embedder).
  
  Example usage:
  
  .. code-block:: python
  
     # down-stream inference
     basic_texts = [
         {"text": "Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot sei"},
         {"text": "Martin Müller spielt Handball in Berlin"},
     ]
     model = Inferencer.load(your_model_dir)
     model.inference_from_dicts(dicts=basic_texts)
     # LM embeddings
     model = Inferencer.load(your_model_dir, extraction_strategy="cls_token", extraction_layer=-1)
     model.inference_from_dicts(dicts=basic_texts)
docstr: Load an Inferencer incl. all relevant components (model, tokenizer, processor ...) either by
  
  1. specifying a public name from transformers' model hub (https://huggingface.co/models)
  2. or pointing to a local directory it is saved in.
  
  :param model_name_or_path: Local directory or public name of the model to load.
  :type model_name_or_path: str
  :param revision: The version of model to use from the HuggingFace model hub. Can be tag name, branch name, or commit hash.
  :type revision: str
  :param batch_size: Number of samples computed once per batch
  :type batch_size: int
  :param gpu: If GPU shall be used
  :type gpu: bool
  :param task_type: Type of task the model should be used for. Currently supporting:
                    "embeddings", "question_answering", "text_classification", "ner". More coming soon...
  :param task_type: str
  :param strict: whether to strictly enforce that the keys loaded from saved model match the ones in
                 the PredictionHead (see torch.nn.module.load_state_dict()).
                 Set to `False` for backwards compatibility with PHs saved with older version of FARM.
  :type strict: bool
  :param max_seq_len: maximum length of one text sample
  :type max_seq_len: int
  :param doc_stride: Only QA: When input text is longer than max_seq_len it gets split into parts, strided by doc_stride
  :type doc_stride: int
  :param extraction_strategy: Strategy to extract vectors. Choices: 'cls_token' (sentence vector), 'reduce_mean'
                         (sentence vector), reduce_max (sentence vector), 'per_token' (individual token vectors)
  :type extraction_strategy: str
  :param extraction_layer: number of layer from which the embeddings shall be extracted. Default: -1 (very last layer).
  :type extraction_layer: int
  :param s3e_stats: Stats of a fitted S3E model as returned by `fit_s3e_on_corpus()`
                    (only needed for task_type="embeddings" and extraction_strategy = "s3e")
  :type s3e_stats: dict
  :param num_processes: the number of processes for `multiprocessing.Pool`. Set to value of 0 to disable
                        multiprocessing. Set to None to let Inferencer use all CPU cores minus one. If you want to
                        debug the Language Model, you might need to disable multiprocessing!
                        **Warning!** If you use multiprocessing you have to close the
                        `multiprocessing.Pool` again! To do so call
                        :func:`~farm.infer.Inferencer.close_multiprocessing_pool` after you are
                        done using this class. The garbage collector will not do this for you!
  :type num_processes: int
  :param disable_tqdm: Whether to disable tqdm logging (can get very verbose in multiprocessing)
  :type disable_tqdm: bool
  :param tokenizer_class: (Optional) Name of the tokenizer class to load (e.g. `BertTokenizer`)
  :type tokenizer_class: str
  :param use_fast: (Optional, True by default) Indicate if FARM should try to load the fast version of the tokenizer (True) or
      use the Python one (False).
  :type use_fast: bool
  :param tokenizer_args: (Optional) Will be passed to the Tokenizer ``__init__`` method.
      See https://huggingface.co/transformers/main_classes/tokenizer.html and detailed tokenizer documentation
      on `Hugging Face Transformers <https://huggingface.co/transformers/>`_.
  :type tokenizer_args: dict
  :param multithreading_rust: Whether to allow multithreading in Rust, e.g. for FastTokenizers.
                              Note: Enabling multithreading in Rust AND multiprocessing in python might cause
                              deadlocks.
  :type multithreading_rust: bool
  :param dummy_ph: If True, methods of the prediction head will be replaced
                       with a dummy method. This is used to isolate lm run time from ph run time.
  :type dummy_ph: bool
  :param benchmarking: If True, a benchmarking object will be initialised within the class and
                       certain parts of the code will be timed for benchmarking. Should be kept
                       False if not benchmarking since these timing checkpoints require synchronization
                       of the asynchronous Pytorch operations and may slow down the model.
  :type benchmarking: bool
  :return: An instance of the Inferencer.
docstr_tok: ["Load","an","Inferencer","incl",".","all","relevant","components","(","model",",","tokenizer",",","processor","...",")","either","by","1.","specifying","a","public","name","from","transformers","'","model","hub","(","https",":","\/\/huggingface.co\/models",")","2.","or","pointing","to","a","local","directory","it","is","saved","in",".",":","param","model_name_or_path",":","Local","directory","or","public","name","of","the","model","to","load",".",":","type","model_name_or_path",":","str",":","param","revision",":","The","version","of","model","to","use","from","the","HuggingFace","model","hub",".","Can","be","tag","name",",","branch","name",",","or","commit","hash",".",":","type","revision",":","str",":","param","batch_size",":","Number","of","samples","computed","once","per","batch",":","type","batch_size",":","int",":","param","gpu",":","If","GPU","shall","be","used",":","type","gpu",":","bool",":","param","task_type",":","Type","of","task","the","model","should","be","used","for",".","Currently","supporting",":","``","embeddings","''",",","``","question_answering","''",",","``","text_classification","''",",","``","ner","''",".","More","coming","soon","...",":","param","task_type",":","str",":","param","strict",":","whether","to","strictly","enforce","that","the","keys","loaded","from","saved","model","match","the","ones","in","the","PredictionHead","(","see","torch.nn.module.load_state_dict","(",")",")",".","Set","to","`","False","`","for","backwards","compatibility","with","PHs","saved","with","older","version","of","FARM",".",":","type","strict",":","bool",":","param","max_seq_len",":","maximum","length","of","one","text","sample",":","type","max_seq_len",":","int",":","param","doc_stride",":","Only","QA",":","When","input","text","is","longer","than","max_seq_len","it","gets","split","into","parts",",","strided","by","doc_stride",":","type","doc_stride",":","int",":","param","extraction_strategy",":","Strategy","to","extract","vectors",".","Choices",":","'cls_token","'","(","sentence","vector",")",",","'reduce_mean'","(","sentence","vector",")",",","reduce_max","(","sentence","vector",")",",","'per_token","'","(","individual","token","vectors",")",":","type","extraction_strategy",":","str",":","param","extraction_layer",":","number","of","layer","from","which","the","embeddings","shall","be","extracted",".","Default",":","-1","(","very","last","layer",")",".",":","type","extraction_layer",":","int",":","param","s3e_stats",":","Stats","of","a","fitted","S3E","model","as","returned","by","`","fit_s3e_on_corpus","(",")","`","(","only","needed","for","task_type=","''","embeddings","''","and","extraction_strategy","=","``","s3e","''",")",":","type","s3e_stats",":","dict",":","param","num_processes",":","the","number","of","processes","for","`","multiprocessing.Pool","`",".","Set","to","value","of","0","to","disable","multiprocessing",".","Set","to","None","to","let","Inferencer","use","all","CPU","cores","minus","one",".","If","you","want","to","debug","the","Language","Model",",","you","might","need","to","disable","multiprocessing","!","*","*","Warning","!","*","*","If","you","use","multiprocessing","you","have","to","close","the","`","multiprocessing.Pool","`","again","!","To","do","so","call",":","func",":","`","~farm.infer.Inferencer.close_multiprocessing_pool","`","after","you","are","done","using","this","class",".","The","garbage","collector","will","not","do","this","for","you","!",":","type","num_processes",":","int",":","param","disable_tqdm",":","Whether","to","disable","tqdm","logging","(","can","get","very","verbose","in","multiprocessing",")",":","type","disable_tqdm",":","bool",":","param","tokenizer_class",":","(","Optional",")","Name","of","the","tokenizer","class","to","load","(","e.g",".","`","BertTokenizer","`",")",":","type","tokenizer_class",":","str",":","param","use_fast",":","(","Optional",",","True","by","default",")","Indicate","if","FARM","should","try","to","load","the","fast","version","of","the","tokenizer","(","True",")","or","use","the","Python","one","(","False",")",".",":","type","use_fast",":","bool",":","param","tokenizer_args",":","(","Optional",")","Will","be","passed","to","the","Tokenizer","``","__init__","``","method",".","See","https",":","\/\/huggingface.co\/transformers\/main_classes\/tokenizer.html","and","detailed","tokenizer","documentation","on","`","Hugging","Face","Transformers","<","https",":","\/\/huggingface.co\/transformers\/",">","`","_",".",":","type","tokenizer_args",":","dict",":","param","multithreading_rust",":","Whether","to","allow","multithreading","in","Rust",",","e.g",".","for","FastTokenizers",".","Note",":","Enabling","multithreading","in","Rust","AND","multiprocessing","in","python","might","cause","deadlocks",".",":","type","multithreading_rust",":","bool",":","param","dummy_ph",":","If","True",",","methods","of","the","prediction","head","will","be","replaced","with","a","dummy","method",".","This","is","used","to","isolate","lm","run","time","from","ph","run","time",".",":","type","dummy_ph",":","bool",":","param","benchmarking",":","If","True",",","a","benchmarking","object","will","be","initialised","within","the","class","and","certain","parts","of","the","code","will","be","timed","for","benchmarking",".","Should","be","kept","False","if","not","benchmarking","since","these","timing","checkpoints","require","synchronization","of","the","asynchronous","Pytorch","operations","and","may","slow","down","the","model",".",":","type","benchmarking",":","bool",":","return",":","An","instance","of","the","Inferencer","."]
code: def load(
      cls,
      model_name_or_path,
      revision=None,
      batch_size=4,
      gpu=False,
      task_type=None,
      return_class_probs=False,
      strict=True,
      max_seq_len=256,
      doc_stride=128,
      extraction_layer=None,
      extraction_strategy=None,
      s3e_stats=None,
      num_processes=None,
      disable_tqdm=False,
      tokenizer_class=None,
      use_fast=True,
      tokenizer_args=None,
      multithreading_rust=True,
      dummy_ph=False,
      benchmarking=False,
      **kwargs,
  ):
      """
      Load an Inferencer incl. all relevant components (model, tokenizer, processor ...) either by
  
      1. specifying a public name from transformers' model hub (https://huggingface.co/models)
      2. or pointing to a local directory it is saved in.
  
      :param model_name_or_path: Local directory or public name of the model to load.
      :type model_name_or_path: str
      :param revision: The version of model to use from the HuggingFace model hub. Can be tag name, branch name, or commit hash.
      :type revision: str
      :param batch_size: Number of samples computed once per batch
      :type batch_size: int
      :param gpu: If GPU shall be used
      :type gpu: bool
      :param task_type: Type of task the model should be used for. Currently supporting:
                        "embeddings", "question_answering", "text_classification", "ner". More coming soon...
      :param task_type: str
      :param strict: whether to strictly enforce that the keys loaded from saved model match the ones in
                     the PredictionHead (see torch.nn.module.load_state_dict()).
                     Set to `False` for backwards compatibility with PHs saved with older version of FARM.
      :type strict: bool
      :param max_seq_len: maximum length of one text sample
      :type max_seq_len: int
      :param doc_stride: Only QA: When input text is longer than max_seq_len it gets split into parts, strided by doc_stride
      :type doc_stride: int
      :param extraction_strategy: Strategy to extract vectors. Choices: 'cls_token' (sentence vector), 'reduce_mean'
                             (sentence vector), reduce_max (sentence vector), 'per_token' (individual token vectors)
      :type extraction_strategy: str
      :param extraction_layer: number of layer from which the embeddings shall be extracted. Default: -1 (very last layer).
      :type extraction_layer: int
      :param s3e_stats: Stats of a fitted S3E model as returned by `fit_s3e_on_corpus()`
                        (only needed for task_type="embeddings" and extraction_strategy = "s3e")
      :type s3e_stats: dict
      :param num_processes: the number of processes for `multiprocessing.Pool`. Set to value of 0 to disable
                            multiprocessing. Set to None to let Inferencer use all CPU cores minus one. If you want to
                            debug the Language Model, you might need to disable multiprocessing!
                            **Warning!** If you use multiprocessing you have to close the
                            `multiprocessing.Pool` again! To do so call
                            :func:`~farm.infer.Inferencer.close_multiprocessing_pool` after you are
                            done using this class. The garbage collector will not do this for you!
      :type num_processes: int
      :param disable_tqdm: Whether to disable tqdm logging (can get very verbose in multiprocessing)
      :type disable_tqdm: bool
      :param tokenizer_class: (Optional) Name of the tokenizer class to load (e.g. `BertTokenizer`)
      :type tokenizer_class: str
      :param use_fast: (Optional, True by default) Indicate if FARM should try to load the fast version of the tokenizer (True) or
          use the Python one (False).
      :type use_fast: bool
      :param tokenizer_args: (Optional) Will be passed to the Tokenizer ``__init__`` method.
          See https://huggingface.co/transformers/main_classes/tokenizer.html and detailed tokenizer documentation
          on `Hugging Face Transformers <https://huggingface.co/transformers/>`_.
      :type tokenizer_args: dict
      :param multithreading_rust: Whether to allow multithreading in Rust, e.g. for FastTokenizers.
                                  Note: Enabling multithreading in Rust AND multiprocessing in python might cause
                                  deadlocks.
      :type multithreading_rust: bool
      :param dummy_ph: If True, methods of the prediction head will be replaced
                           with a dummy method. This is used to isolate lm run time from ph run time.
      :type dummy_ph: bool
      :param benchmarking: If True, a benchmarking object will be initialised within the class and
                           certain parts of the code will be timed for benchmarking. Should be kept
                           False if not benchmarking since these timing checkpoints require synchronization
                           of the asynchronous Pytorch operations and may slow down the model.
      :type benchmarking: bool
      :return: An instance of the Inferencer.
  
      """
      if tokenizer_args is None:
          tokenizer_args = {}
  
      device, n_gpu = initialize_device_settings(
          use_cuda=gpu, local_rank=-1, use_amp=None
      )
      name = os.path.basename(model_name_or_path)
  
      # a) either from local dir
      if os.path.exists(model_name_or_path):
          model = BaseAdaptiveModel.load(
              load_dir=model_name_or_path, device=device, strict=strict
          )
          if task_type == "embeddings":
              processor = InferenceProcessor.load_from_dir(
                  model_name_or_path
              )
          else:
              processor = Processor.load_from_dir(model_name_or_path)
  
      # b) or from remote transformers model hub
      else:
          if not task_type:
              raise ValueError(
                  "Please specify the 'task_type' of the model you want to load from transformers. "
                  "Valid options for arg `task_type`:"
                  "'question_answering', 'embeddings', 'text_classification', 'ner'"
              )
  
          model = AdaptiveModel.convert_from_transformers(
              model_name_or_path,
              revision=revision,
              device=device,
              task_type=task_type,
              **kwargs,
          )
          processor = Processor.convert_from_transformers(
              model_name_or_path,
              revision=revision,
              task_type=task_type,
              max_seq_len=max_seq_len,
              doc_stride=doc_stride,
              tokenizer_class=tokenizer_class,
              tokenizer_args=tokenizer_args,
              use_fast=use_fast,
              **kwargs,
          )
  
      # override processor attributes loaded from config or HF with inferencer params
      processor.max_seq_len = max_seq_len
      processor.multithreading_rust = multithreading_rust
      if hasattr(processor, "doc_stride"):
          assert doc_stride < max_seq_len, (
              "doc_stride is longer than max_seq_len. This means that there will be gaps "
              "as the passage windows slide, causing the model to skip over parts of the document. "
              "Please set a lower value for doc_stride (Suggestions: doc_stride=128, max_seq_len=384) "
          )
          processor.doc_stride = doc_stride
  
      return cls(
          model,
          processor,
          task_type=task_type,
          batch_size=batch_size,
          gpu=gpu,
          name=name,
          return_class_probs=return_class_probs,
          extraction_strategy=extraction_strategy,
          extraction_layer=extraction_layer,
          s3e_stats=s3e_stats,
          num_processes=num_processes,
          disable_tqdm=disable_tqdm,
          benchmarking=benchmarking,
          dummy_ph=dummy_ph,
      )
  
imports: import logging
  import multiprocessing
  import os
  from functools import partial
  import warnings
  import torch
  from torch.utils.data.sampler import SequentialSampler
  from tqdm import tqdm
  from typing import Generator, List, Union
  from farm.data_handler.dataloader import NamedDataLoader
  from farm.data_handler.processor import Processor, InferenceProcessor, NaturalQuestionsProcessor
  from farm.data_handler.utils import grouper
  from farm.data_handler.inputs import QAInput
  from farm.modeling.adaptive_model import AdaptiveModel, BaseAdaptiveModel, ONNXAdaptiveModel
  from farm.modeling.optimization import optimize_model
  from farm.utils import initialize_device_settings, MLFlowLogger
  from farm.utils import set_all_seeds, calc_chunksize, log_ascii_workers, Benchmarker
  from farm.modeling.predictions import QAPred
label_desc: Use image node_id 2 for calling the Inferencer obj's underlying member method code with example usage: obj.load(cls, model_name_or_path, revision, batch_size, gpu, task_type, return_class_probs, strict, max_seq_len, doc_stride, extraction_layer, extraction_strategy, s3e_stats, num_processes, disable_tqdm, tokenizer_class, use_fast, tokenizer_args, multithreading_rust, dummy_ph, benchmarking) and returns: cls