class_docstr: This worker will take tensors from the request queue, do some computation,
  and then return the result back in the response queue.
class_docstr_tok: ["This","worker","will","take","tensors","from","the","request","queue",",","do","some","computation",",","and","then","return","the","result","back","in","the","response","queue","."]
code: def __init__(
      self,
      metrics_dict,
      request_queue,
      response_queue,
      read_requests_event,
      batch_size,
      num_workers,
      model_dir=".",
      compile_model=True,
  ):
      super().__init__()
      self.device = "cuda:0"
      self.metrics_dict = metrics_dict
      self.request_queue = request_queue
      self.response_queue = response_queue
      self.read_requests_event = read_requests_event
      self.batch_size = batch_size
      self.num_workers = num_workers
      self.model_dir = model_dir
      self.compile_model = compile_model
      self._setup_complete = False
      self.h2d_stream = torch.cuda.Stream()
      self.d2h_stream = torch.cuda.Stream()
      # maps thread_id to the cuda.Stream associated with that worker thread
      self.stream_map = dict()
  
imports: import argparse
  import asyncio
  import os.path
  import subprocess
  import threading
  import time
  from concurrent.futures import ThreadPoolExecutor
  from queue import Empty
  import numpy
  import pandas
  import torch
  import torch.multiprocessing
label_desc: Use image node_id 1 to create a new BackendWorker object  with example: obj = BackendWorker(metrics_dict, request_queue, response_queue, read_requests_event, batch_size, num_workers, model_dir, compile_model)