class_docstr: An LLM engine that receives requests and generates texts.
  
  This is the main class for the vLLM engine. It receives requests
  from clients and generates texts from the LLM. It includes a tokenizer, a
  language model (possibly distributed across multiple GPUs), and GPU memory
  space allocated for intermediate states (aka KV cache). This class utilizes
  iteration-level scheduling and efficient memory management to maximize the
  serving throughput.
  
  The `LLM` class wraps this class for offline batched inference and the
  `AsyncLLMEngine` class wraps this class for online serving.
  
  NOTE: The config arguments are derived from the `EngineArgs` class. For the
  comprehensive list of arguments, see `EngineArgs`.
  
  Args:
      model_config: The configuration related to the LLM model.
      cache_config: The configuration related to the KV cache memory
          management.
      parallel_config: The configuration related to distributed execution.
      scheduler_config: The configuration related to the request scheduler.
      distributed_init_method: The initialization method for distributed
          execution. See `torch.distributed.init_process_group` for details.
      placement_group: Ray placement group for distributed execution.
          Required for distributed execution.
      log_stats: Whether to log statistics.
docstr: Add a request to the engine's request pool.
  
  The request is added to the request pool and will be processed by the
  scheduler as `engine.step()` is called. The exact scheduling policy is
  determined by the scheduler.
  
  Args:
      request_id: The unique ID of the request.
      prompt: The prompt string. Can be None if prompt_token_ids is
          provided.
      sampling_params: The sampling parameters for text generation.
      prompt_token_ids: The token IDs of the prompt. If None, we
          use the tokenizer to convert the prompts to token IDs.
      arrival_time: The arrival time of the request. If None, we use
          the current monotonic time.
docstr_tok: ["Add","a","request","to","the","engine","'s","request","pool",".","The","request","is","added","to","the","request","pool","and","will","be","processed","by","the","scheduler","as","`","engine.step","(",")","`","is","called",".","The","exact","scheduling","policy","is","determined","by","the","scheduler",".","Args",":","request_id",":","The","unique","ID","of","the","request",".","prompt",":","The","prompt","string",".","Can","be","None","if","prompt_token_ids","is","provided",".","sampling_params",":","The","sampling","parameters","for","text","generation",".","prompt_token_ids",":","The","token","IDs","of","the","prompt",".","If","None",",","we","use","the","tokenizer","to","convert","the","prompts","to","token","IDs",".","arrival_time",":","The","arrival","time","of","the","request",".","If","None",",","we","use","the","current","monotonic","time","."]
code: def add_request(
      self,
      request_id: str,
      prompt: Optional[str],
      sampling_params: SamplingParams,
      prompt_token_ids: Optional[List[int]] = None,
      arrival_time: Optional[float] = None,
  ) -> None:
      """Add a request to the engine's request pool.
  
      The request is added to the request pool and will be processed by the
      scheduler as `engine.step()` is called. The exact scheduling policy is
      determined by the scheduler.
  
      Args:
          request_id: The unique ID of the request.
          prompt: The prompt string. Can be None if prompt_token_ids is
              provided.
          sampling_params: The sampling parameters for text generation.
          prompt_token_ids: The token IDs of the prompt. If None, we
              use the tokenizer to convert the prompts to token IDs.
          arrival_time: The arrival time of the request. If None, we use
              the current monotonic time.
      """
      if arrival_time is None:
          arrival_time = time.monotonic()
      if prompt_token_ids is None:
          invalidInputError(
              prompt is not None, "Prompt should not be None"
          )
          prompt_token_ids = self.tokenizer.encode(prompt)
  
      # Create the sequences.
      seq_id = next(self.seq_counter)
      seq = Sequence(seq_id, prompt, prompt_token_ids)
  
      # Create the sequence group.
      seq_group = SequenceGroup(
          request_id, [seq], sampling_params, arrival_time
      )
  
      # Add the sequence group to the scheduler.
      self.scheduler.add_seq_group(seq_group)
  
imports: import time
  from typing import TYPE_CHECKING, Any, Iterable, List, Optional, Tuple, Union, Dict
  from bigdl.llm.vllm.config import ModelConfig, SchedulerConfig
  from bigdl.llm.vllm.core.scheduler import SchedulerOutputs, FixedWindowScheduler
  from bigdl.llm.vllm.engine.arg_utils import EngineArgs
  from bigdl.llm.vllm.logger import init_logger
  from bigdl.llm.vllm.outputs import RequestOutput
  from bigdl.llm.vllm.sampling_params import SamplingParams
  from bigdl.llm.vllm.sequence import SamplerOutput, Sequence, SequenceGroup, SequenceGroupMetadata, SequenceStatus, SequenceOutputs
  from bigdl.llm.vllm.transformers_utils.tokenizer import get_tokenizer, detokenize_incrementally
  from bigdl.llm.vllm.utils import Counter
  from bigdl.llm.utils.common import invalidInputError
label_desc: Use image node_id 5 for calling the LLMEngine obj's underlying member method code with example usage: obj.add_request(request_id, prompt, sampling_params, prompt_token_ids, arrival_time) without return types