class_docstr: Generic base class for remotely-hosted models.
  
  Implements concurrent request batching; subclass need only implement
  predict_minibatch() and max_minibatch_size().
  
  If subclass overrides __init__, it should be sure to call super().__init__()
  to set up the threadpool.
docstr: Maximum minibatch size for this model. Subclass can override this.
docstr_tok: ["Maximum","minibatch","size","for","this","model",".","Subclass","can","override","this","."]
code: def max_minibatch_size(self) -> int:
      """Maximum minibatch size for this model. Subclass can override this."""
      return 1
  
imports: import abc
  from collections.abc import Iterable, Iterator
  import inspect
  import itertools
  import multiprocessing
  from typing import Optional, Union
  from absl import logging
  from lit_nlp.api import dataset
  from lit_nlp.api import types
  from lit_nlp.lib import utils
  import numpy
label_desc: Use image node_id 3 for calling the BatchedRemoteModel obj's underlying member method code with example usage: obj.max_minibatch_size() and returns: int