class_docstr: Generic base class for the batched model.
  
  Subclass needs to implement predict_minibatch() and optionally
  max_minibatch_size().
docstr: Run prediction on a dataset.
  
  This uses minibatch inference for efficiency, but yields per-example output.
  
  This will also copy some NumPy arrays if they look like slices of a larger
  tensor. This adds some overhead, but reduces memory leaks by allowing the
  source tensor (which may be a large padded matrix) to be garbage collected.
  
  Args:
    inputs: iterable of input dicts
    **kw: additional kwargs passed to predict_minibatch()
  
  Returns:
    model outputs, for each input
docstr_tok: ["Run","prediction","on","a","dataset",".","This","uses","minibatch","inference","for","efficiency",",","but","yields","per-example","output",".","This","will","also","copy","some","NumPy","arrays","if","they","look","like","slices","of","a","larger","tensor",".","This","adds","some","overhead",",","but","reduces","memory","leaks","by","allowing","the","source","tensor","(","which","may","be","a","large","padded","matrix",")","to","be","garbage","collected",".","Args",":","inputs",":","iterable","of","input","dicts","*","*","kw",":","additional","kwargs","passed","to","predict_minibatch","(",")","Returns",":","model","outputs",",","for","each","input"]
code: def predict(
      self, inputs: Iterable[JsonDict], **kw
  ) -> Iterable[JsonDict]:
      """Run prediction on a dataset.
  
      This uses minibatch inference for efficiency, but yields per-example output.
  
      This will also copy some NumPy arrays if they look like slices of a larger
      tensor. This adds some overhead, but reduces memory leaks by allowing the
      source tensor (which may be a large padded matrix) to be garbage collected.
  
      Args:
        inputs: iterable of input dicts
        **kw: additional kwargs passed to predict_minibatch()
  
      Returns:
        model outputs, for each input
      """
      results = self.batched_predict(inputs, **kw)
      results = (scrub_numpy_refs(res) for res in results)
      return results
  
imports: import abc
  from collections.abc import Iterable, Iterator
  import inspect
  import itertools
  import multiprocessing
  from typing import Optional, Union
  from absl import logging
  from lit_nlp.api import dataset
  from lit_nlp.api import types
  from lit_nlp.lib import utils
  import numpy
label_desc: Use image node_id 4 for calling the BatchedModel obj's underlying member method code with example usage: obj.predict(inputs) and returns: results