class_docstr: Generic base class for remotely-hosted models.
  
  Implements concurrent request batching; subclass need only implement
  predict_minibatch() and max_minibatch_size().
  
  If subclass overrides __init__, it should be sure to call super().__init__()
  to set up the threadpool.
code: def __init__(
      self,
      max_concurrent_requests: int = 4,
      max_qps: Union[int, float] = 25,
  ):
      # Use a local thread pool for concurrent requests, so we can keep the server
      # busy during network transit time and local pre/post-processing.
      self._max_qps = max_qps
      self._pool = multiprocessing.pool.ThreadPool(
          max_concurrent_requests
      )
  
imports: import abc
  from collections.abc import Iterable, Iterator
  import inspect
  import itertools
  import multiprocessing
  from typing import Optional, Union
  from absl import logging
  from lit_nlp.api import dataset
  from lit_nlp.api import types
  from lit_nlp.lib import utils
  import numpy
label_desc: Use image node_id 1 to create a new BatchedRemoteModel object from inherited base classes: Model with example: obj = BatchedRemoteModel(max_concurrent_requests, max_qps)