class_docstr: Loads a saved AdaptiveModel/ONNXAdaptiveModel from disk and runs it in inference mode. Can be used for a
  model with prediction head (down-stream predictions) and without (using LM as embedder).
  
  Example usage:
  
  .. code-block:: python
  
     # down-stream inference
     basic_texts = [
         {"text": "Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot sei"},
         {"text": "Martin Müller spielt Handball in Berlin"},
     ]
     model = Inferencer.load(your_model_dir)
     model.inference_from_dicts(dicts=basic_texts)
     # LM embeddings
     model = Inferencer.load(your_model_dir, extraction_strategy="cls_token", extraction_layer=-1)
     model.inference_from_dicts(dicts=basic_texts)
docstr: Runs down-stream inference on samples created from input dictionaries.
  The format of the input `dicts` depends on the task:
  
  * QA (FARM style): [{"questions": ["What is X?"], "text":  "Some context containing the answer"}]
  * Classification / NER / embeddings: [{"text": "Some input text"}]
  
  Inferencer has a high performance non-blocking streaming mode for large scale inference use cases. With this
  mode, the dicts parameter can optionally be a Python generator object that yield dicts, thus avoiding loading
  dicts in memory. The inference_from_dicts() method returns a generator that yield predictions. To use streaming,
  set the streaming param to True and determine optimal multiprocessing_chunksize by performing speed benchmarks.
  
  
  :param dicts: Samples to run inference on provided as a list(or a generator object) of dicts.
                One dict per sample.
  :type dicts: iter(dict)
  :param return_json: Whether the output should be in a json appropriate format. If False, it returns the prediction
                      object where applicable, else it returns PredObj.to_json()
  :type return_json: bool
  :return: dict of predictions
  :param multiprocessing_chunksize: number of dicts to put together in one chunk and feed to one process
                                    (only relevant if you do multiprocessing)
  :type multiprocessing_chunksize: int
  :param streaming: return a Python generator object that yield results as they get computed, instead of blocking
                    for all the results. To use streaming, the dicts parameter must be a generator and
                    num_processes argument must be set. This mode can be useful to implement large scale
                    non-blocking inference pipelines.
  :type streaming: bool
  
  :return: an iterator(list or generator) of predictions
  :rtype: iter
docstr_tok: ["Runs","down-stream","inference","on","samples","created","from","input","dictionaries",".","The","format","of","the","input","`","dicts","`","depends","on","the","task",":","*","QA","(","FARM","style",")",":","[","{","``","questions","''",":","[","``","What","is","X","?","``","]",",","``","text","''",":","``","Some","context","containing","the","answer","''","}","]","*","Classification","\/","NER","\/","embeddings",":","[","{","``","text","''",":","``","Some","input","text","''","}","]","Inferencer","has","a","high","performance","non-blocking","streaming","mode","for","large","scale","inference","use","cases",".","With","this","mode",",","the","dicts","parameter","can","optionally","be","a","Python","generator","object","that","yield","dicts",",","thus","avoiding","loading","dicts","in","memory",".","The","inference_from_dicts","(",")","method","returns","a","generator","that","yield","predictions",".","To","use","streaming",",","set","the","streaming","param","to","True","and","determine","optimal","multiprocessing_chunksize","by","performing","speed","benchmarks",".",":","param","dicts",":","Samples","to","run","inference","on","provided","as","a","list","(","or","a","generator","object",")","of","dicts",".","One","dict","per","sample",".",":","type","dicts",":","iter","(","dict",")",":","param","return_json",":","Whether","the","output","should","be","in","a","json","appropriate","format",".","If","False",",","it","returns","the","prediction","object","where","applicable",",","else","it","returns","PredObj.to_json","(",")",":","type","return_json",":","bool",":","return",":","dict","of","predictions",":","param","multiprocessing_chunksize",":","number","of","dicts","to","put","together","in","one","chunk","and","feed","to","one","process","(","only","relevant","if","you","do","multiprocessing",")",":","type","multiprocessing_chunksize",":","int",":","param","streaming",":","return","a","Python","generator","object","that","yield","results","as","they","get","computed",",","instead","of","blocking","for","all","the","results",".","To","use","streaming",",","the","dicts","parameter","must","be","a","generator","and","num_processes","argument","must","be","set",".","This","mode","can","be","useful","to","implement","large","scale","non-blocking","inference","pipelines",".",":","type","streaming",":","bool",":","return",":","an","iterator","(","list","or","generator",")","of","predictions",":","rtype",":","iter"]
code: def inference_from_dicts(
      self,
      dicts,
      return_json=True,
      multiprocessing_chunksize=None,
      streaming=False,
  ):
      """
      Runs down-stream inference on samples created from input dictionaries.
      The format of the input `dicts` depends on the task:
  
      * QA (FARM style): [{"questions": ["What is X?"], "text":  "Some context containing the answer"}]
      * Classification / NER / embeddings: [{"text": "Some input text"}]
  
      Inferencer has a high performance non-blocking streaming mode for large scale inference use cases. With this
      mode, the dicts parameter can optionally be a Python generator object that yield dicts, thus avoiding loading
      dicts in memory. The inference_from_dicts() method returns a generator that yield predictions. To use streaming,
      set the streaming param to True and determine optimal multiprocessing_chunksize by performing speed benchmarks.
  
  
      :param dicts: Samples to run inference on provided as a list(or a generator object) of dicts.
                    One dict per sample.
      :type dicts: iter(dict)
      :param return_json: Whether the output should be in a json appropriate format. If False, it returns the prediction
                          object where applicable, else it returns PredObj.to_json()
      :type return_json: bool
      :return: dict of predictions
      :param multiprocessing_chunksize: number of dicts to put together in one chunk and feed to one process
                                        (only relevant if you do multiprocessing)
      :type multiprocessing_chunksize: int
      :param streaming: return a Python generator object that yield results as they get computed, instead of blocking
                        for all the results. To use streaming, the dicts parameter must be a generator and
                        num_processes argument must be set. This mode can be useful to implement large scale
                        non-blocking inference pipelines.
      :type streaming: bool
  
      :return: an iterator(list or generator) of predictions
      :rtype: iter
      """
  
      # whether to aggregate predictions across different samples (e.g. for QA on long texts)
      # TODO remove or adjust after implmenting input objects properly
      # if set(dicts[0].keys()) == {"qas", "context"}:
      #     warnings.warn("QA Input dictionaries with [qas, context] as keys will be deprecated in the future",
      #                   DeprecationWarning)
  
      aggregate_preds = False
      if len(self.model.prediction_heads) > 0:
          aggregate_preds = hasattr(
              self.model.prediction_heads[0], "aggregate_preds"
          )
  
      if (
          self.process_pool is None
      ):  # multiprocessing disabled (helpful for debugging or using in web frameworks)
          predictions = self._inference_without_multiprocessing(
              dicts, return_json, aggregate_preds
          )
          return predictions
      else:  # use multiprocessing for inference
          # Calculate values of multiprocessing_chunksize and num_processes if not supplied in the parameters.
          # The calculation of the values is based on whether streaming mode is enabled. This is only for speed
          # optimization and do not impact the results of inference.
          if streaming:
              if multiprocessing_chunksize is None:
                  logger.warning(
                      "Streaming mode is enabled for the Inferencer but multiprocessing_chunksize is not "
                      "supplied. Continuing with a default value of 20. Perform benchmarking on your data "
                      "to get the optimal chunksize."
                  )
                  multiprocessing_chunksize = 20
          else:
              if multiprocessing_chunksize is None:
                  _chunk_size, _ = calc_chunksize(len(dicts))
                  multiprocessing_chunksize = _chunk_size
  
          predictions = self._inference_with_multiprocessing(
              dicts,
              return_json,
              aggregate_preds,
              multiprocessing_chunksize,
          )
  
          self.processor.log_problematic(self.problematic_sample_ids)
          # return a generator object if streaming is enabled, else, cast the generator to a list.
          if not streaming and type(predictions) != list:
              return list(predictions)
          else:
              return predictions
  
imports: import logging
  import multiprocessing
  import os
  from functools import partial
  import warnings
  import torch
  from torch.utils.data.sampler import SequentialSampler
  from tqdm import tqdm
  from typing import Generator, List, Union
  from farm.data_handler.dataloader import NamedDataLoader
  from farm.data_handler.processor import Processor, InferenceProcessor, NaturalQuestionsProcessor
  from farm.data_handler.utils import grouper
  from farm.data_handler.inputs import QAInput
  from farm.modeling.adaptive_model import AdaptiveModel, BaseAdaptiveModel, ONNXAdaptiveModel
  from farm.modeling.optimization import optimize_model
  from farm.utils import initialize_device_settings, MLFlowLogger
  from farm.utils import set_all_seeds, calc_chunksize, log_ascii_workers, Benchmarker
  from farm.modeling.predictions import QAPred
label_desc: Use image node_id 7 for calling the Inferencer obj's underlying member method code with example usage: obj.inference_from_dicts(dicts, return_json, multiprocessing_chunksize, streaming) and returns: predictions, list, predictions