class_docstr: Generic base class for remotely-hosted models.
  
  Implements concurrent request batching; subclass need only implement
  predict_minibatch() and max_minibatch_size().
  
  If subclass overrides __init__, it should be sure to call super().__init__()
  to set up the threadpool.
code: def predict(
      self,
      inputs: Iterable[JsonDict],
      *unused_args,
      parallel=True,
      **unused_kwargs,
  ) -> Iterator[JsonDict]:
      batches = utils.batch_iterator(
          inputs, max_batch_size=self.max_minibatch_size()
      )
      batches = utils.rate_limit(batches, self._max_qps)
      if parallel:
          pred_batches = self._pool.imap(
              self.predict_minibatch, batches
          )
      else:  # don't use the threadpool; useful for debugging
          pred_batches = map(self.predict_minibatch, batches)
      return itertools.chain.from_iterable(pred_batches)
  
imports: import abc
  from collections.abc import Iterable, Iterator
  import inspect
  import itertools
  import multiprocessing
  from typing import Optional, Union
  from absl import logging
  from lit_nlp.api import dataset
  from lit_nlp.api import types
  from lit_nlp.lib import utils
  import numpy
label_desc: Use image node_id 2 for calling the BatchedRemoteModel obj's underlying member method code with example usage: obj.predict(inputs) and returns: itertools