class_docstr: Auto configuration.
class_docstr_tok: ["Auto","configuration","."]
docstr: Auto configuration to run with triton server(GPUs).
  
  Args:
     device_ids(`list`):
         list of GPUs, defaults to [0].
     max_batch_size(`int`):
         maximum batch size, defaults to None, and it will be auto-generated by triton.
     batch_latency_micros(`int`):
         time to the request, in microseconds, defaults to None, and it will auto-generated by triton.
     num_instances_per_device(`int`):
         the number of instances per device, defaults to 1.
     preferred_batch_size(`list`):
         preferred batch sizes for dynamic batching, defaults to None, and it will be auto-generated by triton.
  
  Examples:
      >>> from towhee import pipe, AutoConfig
      >>> p = (pipe.input('url')
      ...          .map('url', 'image', ops.image_decode.cv2())
      ...          .map('image', 'vec', ops.image_embedding.timm(model_name='resnet50'), config=AutoConfig.TritonGPUConfig())
      ...          .output('vec')
      ... )
  
      You can also to set the configuration:
  
      >>> from towhee import pipe, AutoConfig
      >>> config = AutoConfig.TritonGPUConfig(device_ids=[0, 1],
      ...                                     num_instances_per_device=3,
      ...                                     max_batch_size=128,
      ...                                     batch_latency_micros=100000,
      ...                                     preferred_batch_size=[8, 16])
      >>> p = (pipe.input('url')
      ...          .map('url', 'image', ops.image_decode.cv2())
      ...          .map('image', 'vec', ops.image_embedding.timm(model_name='resnet50'), config=config)
      ...          .output('vec')
      ... )
docstr_tok: ["Auto","configuration","to","run","with","triton","server","(","GPUs",")",".","Args",":","device_ids","(","`","list","`",")",":","list","of","GPUs",",","defaults","to","[","0","]",".","max_batch_size","(","`","int","`",")",":","maximum","batch","size",",","defaults","to","None",",","and","it","will","be","auto-generated","by","triton",".","batch_latency_micros","(","`","int","`",")",":","time","to","the","request",",","in","microseconds",",","defaults","to","None",",","and","it","will","auto-generated","by","triton",".","num_instances_per_device","(","`","int","`",")",":","the","number","of","instances","per","device",",","defaults","to","1.","preferred_batch_size","(","`","list","`",")",":","preferred","batch","sizes","for","dynamic","batching",",","defaults","to","None",",","and","it","will","be","auto-generated","by","triton",".","Examples",":",">",">",">","from","towhee","import","pipe",",","AutoConfig",">",">",">","p","=","(","pipe.input","(","'url","'",")","...",".map","(","'url","'",",","'image","'",",","ops.image_decode.cv2","(",")",")","...",".map","(","'image","'",",","'vec","'",",","ops.image_embedding.timm","(","model_name='resnet50","'",")",",","config=AutoConfig.TritonGPUConfig","(",")",")","...",".output","(","'vec","'",")","...",")","You","can","also","to","set","the","configuration",":",">",">",">","from","towhee","import","pipe",",","AutoConfig",">",">",">","config","=","AutoConfig.TritonGPUConfig","(","device_ids=","[","0",",","1","]",",","...","num_instances_per_device=3",",","...","max_batch_size=128",",","...","batch_latency_micros=100000",",","...","preferred_batch_size=","[","8",",","16","]",")",">",">",">","p","=","(","pipe.input","(","'url","'",")","...",".map","(","'url","'",",","'image","'",",","ops.image_decode.cv2","(",")",")","...",".map","(","'image","'",",","'vec","'",",","ops.image_embedding.timm","(","model_name='resnet50","'",")",",","config=config",")","...",".output","(","'vec","'",")","...",")"]
code: def TritonGPUConfig(
      device_ids: list = None,
      num_instances_per_device: int = 1,
      max_batch_size: int = None,
      batch_latency_micros: int = None,
      preferred_batch_size: list = None,
  ):
      """
      Auto configuration to run with triton server(GPUs).
  
      Args:
         device_ids(`list`):
             list of GPUs, defaults to [0].
         max_batch_size(`int`):
             maximum batch size, defaults to None, and it will be auto-generated by triton.
         batch_latency_micros(`int`):
             time to the request, in microseconds, defaults to None, and it will auto-generated by triton.
         num_instances_per_device(`int`):
             the number of instances per device, defaults to 1.
         preferred_batch_size(`list`):
             preferred batch sizes for dynamic batching, defaults to None, and it will be auto-generated by triton.
  
      Examples:
          >>> from towhee import pipe, AutoConfig
          >>> p = (pipe.input('url')
          ...          .map('url', 'image', ops.image_decode.cv2())
          ...          .map('image', 'vec', ops.image_embedding.timm(model_name='resnet50'), config=AutoConfig.TritonGPUConfig())
          ...          .output('vec')
          ... )
  
          You can also to set the configuration:
  
          >>> from towhee import pipe, AutoConfig
          >>> config = AutoConfig.TritonGPUConfig(device_ids=[0, 1],
          ...                                     num_instances_per_device=3,
          ...                                     max_batch_size=128,
          ...                                     batch_latency_micros=100000,
          ...                                     preferred_batch_size=[8, 16])
          >>> p = (pipe.input('url')
          ...          .map('url', 'image', ops.image_decode.cv2())
          ...          .map('image', 'vec', ops.image_embedding.timm(model_name='resnet50'), config=config)
          ...          .output('vec')
          ... )
      """
      if device_ids is None:
          device_ids = [0]
      return nd_conf.TowheeConfig.set_triton_config(
          device_ids=device_ids,
          num_instances_per_device=num_instances_per_device,
          max_batch_size=max_batch_size,
          batch_latency_micros=batch_latency_micros,
          preferred_batch_size=preferred_batch_size,
      )
  
file_path: towhee/towhee/runtime/auto_config.py
imports: import threading
  from functools import wraps
  import contextvars
  import contextlib
  import towhee.runtime.pipeline_loader
  import towhee.runtime.node_config
  from towhee.utils.log import engine_log
label_desc: Use image node_id 7 for calling the AutoConfig obj's underlying member method code with example usage: obj.TritonGPUConfig(device_ids, num_instances_per_device, max_batch_size, batch_latency_micros, preferred_batch_size) and returns: nd_conf