class_docstr: A DJL SageMaker ``Model`` that can be deployed to a SageMaker ``Endpoint``.
docstr: Initialize a DJLModel.
  
  Args:
      model_id (str): This is either the HuggingFace Hub model_id, or the Amazon S3 location
          containing the uncompressed model artifacts (i.e. not a tar.gz file).
          The model artifacts are expected to be in HuggingFace pre-trained model
          format (i.e. model should be loadable from the huggingface transformers
          from_pretrained api, and should also include tokenizer configs if applicable).
      role (str): An AWS IAM role specified with either the name or full ARN. The Amazon
          SageMaker training jobs and APIs that create Amazon SageMaker
          endpoints use this role to access model artifacts. After the endpoint is created,
          the inference code might use the IAM role, if it needs to access an AWS resource.
      djl_version (str): DJL Serving version you want to use for serving your model for
          inference. Defaults to None. If not provided, the latest available version of DJL
          Serving is used. This is not used if ``image_uri`` is provided.
      task (str): The HuggingFace/NLP task you want to launch this model for. Defaults to
          None.
          If not provided, the task will be inferred from the model architecture by DJL.
      dtype (str): The data type to use for loading your model. Accepted values are
          "fp32", "fp16", "bf16", "int8". Defaults to "fp32".
      number_of_partitions (int): The number of GPUs to partition the model across. The
          partitioning strategy is determined by the selected backend. If DeepSpeed is
          selected, this is tensor parallelism.
          If HuggingFace Accelerate is selected, this is a naive sharding strategy
          that splits the model layers across the available resources. Defaults to None. If
          not provided, no model partitioning is done.
      min_workers (int): The minimum number of worker processes. Defaults to None. If not
          provided, dJL Serving will automatically detect the minimum workers.
      max_workers (int): The maximum number of worker processes. Defaults to None. If not
          provided, DJL Serving will automatically detect the maximum workers.
      job_queue_size (int): The request job queue size. Defaults to None. If not specified,
          defaults to 1000.
      parallel_loading (bool): Whether to load model workers in parallel. Defaults to False,
          in which case DJL Serving will load the model workers sequentially to reduce the
          risk of running out of memory. Set to True if you want to reduce model loading
          time and know that peak memory usage will not cause out of memory issues.
      model_loading_timeout (int): The worker model loading timeout in seconds. Defaults to
          None. If not provided, the default is 240 seconds.
      prediction_timeout (int): The worker predict call (handler) timeout in seconds.
          Defaults to None. If not provided, the default is 120 seconds.
      entry_point (str): This can either be the absolute or relative path to the Python source
          file that should be executed as the entry point to model
          hosting, or a python module that is installed in the container. If ``source_dir``
          is specified, then ``entry_point``
          must point to a file located at the root of ``source_dir``. Defaults to None.
      image_uri (str): A docker image URI. Defaults to None. If not specified, a default
          image for DJL Serving will be used based on ``djl_version``. If ``djl_version``
          is not specified, the latest available container version will be used.
      predictor_cls (callable[str, sagemaker.session.Session]): A function to call to create a
          predictor with an endpoint name and SageMaker ``Session``. If specified,
          ``deploy()`` returns
          the result of invoking this function on the created endpoint name.
      **kwargs: Keyword arguments passed to the superclass
          :class:`~sagemaker.model.FrameworkModel` and, subsequently, its
          superclass :class:`~sagemaker.model.Model`.
  
  .. tip::
  
      Instantiating a DJLModel will return an instance of either
      :class:`~sagemaker.djl_inference.DeepSpeedModel` or
      :class:`~sagemaker.djl_inference.HuggingFaceAccelerateModel` based on our framework
      recommendation for the model type.
  
      If you want to use a specific framework to deploy your model with, we recommend
      instantiating that specific
      model class directly. The available framework specific classes are
      :class:`~sagemaker.djl_inference.DeepSpeedModel` or
      :class:`~sagemaker.djl_inference.HuggingFaceAccelerateModel`
docstr_tok: ["Initialize","a","DJLModel",".","Args",":","model_id","(","str",")",":","This","is","either","the","HuggingFace","Hub","model_id",",","or","the","Amazon","S3","location","containing","the","uncompressed","model","artifacts","(","i.e",".","not","a","tar.gz","file",")",".","The","model","artifacts","are","expected","to","be","in","HuggingFace","pre-trained","model","format","(","i.e",".","model","should","be","loadable","from","the","huggingface","transformers","from_pretrained","api",",","and","should","also","include","tokenizer","configs","if","applicable",")",".","role","(","str",")",":","An","AWS","IAM","role","specified","with","either","the","name","or","full","ARN",".","The","Amazon","SageMaker","training","jobs","and","APIs","that","create","Amazon","SageMaker","endpoints","use","this","role","to","access","model","artifacts",".","After","the","endpoint","is","created",",","the","inference","code","might","use","the","IAM","role",",","if","it","needs","to","access","an","AWS","resource",".","djl_version","(","str",")",":","DJL","Serving","version","you","want","to","use","for","serving","your","model","for","inference",".","Defaults","to","None",".","If","not","provided",",","the","latest","available","version","of","DJL","Serving","is","used",".","This","is","not","used","if","``","image_uri","``","is","provided",".","task","(","str",")",":","The","HuggingFace\/NLP","task","you","want","to","launch","this","model","for",".","Defaults","to","None",".","If","not","provided",",","the","task","will","be","inferred","from","the","model","architecture","by","DJL",".","dtype","(","str",")",":","The","data","type","to","use","for","loading","your","model",".","Accepted","values","are","``","fp32","''",",","``","fp16","''",",","``","bf16","''",",","``","int8","''",".","Defaults","to","``","fp32","''",".","number_of_partitions","(","int",")",":","The","number","of","GPUs","to","partition","the","model","across",".","The","partitioning","strategy","is","determined","by","the","selected","backend",".","If","DeepSpeed","is","selected",",","this","is","tensor","parallelism",".","If","HuggingFace","Accelerate","is","selected",",","this","is","a","naive","sharding","strategy","that","splits","the","model","layers","across","the","available","resources",".","Defaults","to","None",".","If","not","provided",",","no","model","partitioning","is","done",".","min_workers","(","int",")",":","The","minimum","number","of","worker","processes",".","Defaults","to","None",".","If","not","provided",",","dJL","Serving","will","automatically","detect","the","minimum","workers",".","max_workers","(","int",")",":","The","maximum","number","of","worker","processes",".","Defaults","to","None",".","If","not","provided",",","DJL","Serving","will","automatically","detect","the","maximum","workers",".","job_queue_size","(","int",")",":","The","request","job","queue","size",".","Defaults","to","None",".","If","not","specified",",","defaults","to","1000.","parallel_loading","(","bool",")",":","Whether","to","load","model","workers","in","parallel",".","Defaults","to","False",",","in","which","case","DJL","Serving","will","load","the","model","workers","sequentially","to","reduce","the","risk","of","running","out","of","memory",".","Set","to","True","if","you","want","to","reduce","model","loading","time","and","know","that","peak","memory","usage","will","not","cause","out","of","memory","issues",".","model_loading_timeout","(","int",")",":","The","worker","model","loading","timeout","in","seconds",".","Defaults","to","None",".","If","not","provided",",","the","default","is","240","seconds",".","prediction_timeout","(","int",")",":","The","worker","predict","call","(","handler",")","timeout","in","seconds",".","Defaults","to","None",".","If","not","provided",",","the","default","is","120","seconds",".","entry_point","(","str",")",":","This","can","either","be","the","absolute","or","relative","path","to","the","Python","source","file","that","should","be","executed","as","the","entry","point","to","model","hosting",",","or","a","python","module","that","is","installed","in","the","container",".","If","``","source_dir","``","is","specified",",","then","``","entry_point","``","must","point","to","a","file","located","at","the","root","of","``","source_dir","``",".","Defaults","to","None",".","image_uri","(","str",")",":","A","docker","image","URI",".","Defaults","to","None",".","If","not","specified",",","a","default","image","for","DJL","Serving","will","be","used","based","on","``","djl_version","``",".","If","``","djl_version","``","is","not","specified",",","the","latest","available","container","version","will","be","used",".","predictor_cls","(","callable","[","str",",","sagemaker.session.Session","]",")",":","A","function","to","call","to","create","a","predictor","with","an","endpoint","name","and","SageMaker","``","Session","``",".","If","specified",",","``","deploy","(",")","``","returns","the","result","of","invoking","this","function","on","the","created","endpoint","name",".","*","*","kwargs",":","Keyword","arguments","passed","to","the","superclass",":","class",":","`","~sagemaker.model.FrameworkModel","`","and",",","subsequently",",","its","superclass",":","class",":","`","~sagemaker.model.Model","`",".","..","tip",":",":","Instantiating","a","DJLModel","will","return","an","instance","of","either",":","class",":","`","~sagemaker.djl_inference.DeepSpeedModel","`","or",":","class",":","`","~sagemaker.djl_inference.HuggingFaceAccelerateModel","`","based","on","our","framework","recommendation","for","the","model","type",".","If","you","want","to","use","a","specific","framework","to","deploy","your","model","with",",","we","recommend","instantiating","that","specific","model","class","directly",".","The","available","framework","specific","classes","are",":","class",":","`","~sagemaker.djl_inference.DeepSpeedModel","`","or",":","class",":","`","~sagemaker.djl_inference.HuggingFaceAccelerateModel","`"]
code: def __init__(
      self,
      model_id: str,
      role: str,
      djl_version: Optional[str] = None,
      task: Optional[str] = None,
      dtype: str = "fp32",
      number_of_partitions: Optional[int] = None,
      min_workers: Optional[int] = None,
      max_workers: Optional[int] = None,
      job_queue_size: Optional[int] = None,
      parallel_loading: bool = False,
      model_loading_timeout: Optional[int] = None,
      prediction_timeout: Optional[int] = None,
      entry_point: Optional[str] = None,
      image_uri: Optional[Union[str, PipelineVariable]] = None,
      predictor_cls: callable = DJLPredictor,
      **kwargs,
  ):
      """Initialize a DJLModel.
  
      Args:
          model_id (str): This is either the HuggingFace Hub model_id, or the Amazon S3 location
              containing the uncompressed model artifacts (i.e. not a tar.gz file).
              The model artifacts are expected to be in HuggingFace pre-trained model
              format (i.e. model should be loadable from the huggingface transformers
              from_pretrained api, and should also include tokenizer configs if applicable).
          role (str): An AWS IAM role specified with either the name or full ARN. The Amazon
              SageMaker training jobs and APIs that create Amazon SageMaker
              endpoints use this role to access model artifacts. After the endpoint is created,
              the inference code might use the IAM role, if it needs to access an AWS resource.
          djl_version (str): DJL Serving version you want to use for serving your model for
              inference. Defaults to None. If not provided, the latest available version of DJL
              Serving is used. This is not used if ``image_uri`` is provided.
          task (str): The HuggingFace/NLP task you want to launch this model for. Defaults to
              None.
              If not provided, the task will be inferred from the model architecture by DJL.
          dtype (str): The data type to use for loading your model. Accepted values are
              "fp32", "fp16", "bf16", "int8". Defaults to "fp32".
          number_of_partitions (int): The number of GPUs to partition the model across. The
              partitioning strategy is determined by the selected backend. If DeepSpeed is
              selected, this is tensor parallelism.
              If HuggingFace Accelerate is selected, this is a naive sharding strategy
              that splits the model layers across the available resources. Defaults to None. If
              not provided, no model partitioning is done.
          min_workers (int): The minimum number of worker processes. Defaults to None. If not
              provided, dJL Serving will automatically detect the minimum workers.
          max_workers (int): The maximum number of worker processes. Defaults to None. If not
              provided, DJL Serving will automatically detect the maximum workers.
          job_queue_size (int): The request job queue size. Defaults to None. If not specified,
              defaults to 1000.
          parallel_loading (bool): Whether to load model workers in parallel. Defaults to False,
              in which case DJL Serving will load the model workers sequentially to reduce the
              risk of running out of memory. Set to True if you want to reduce model loading
              time and know that peak memory usage will not cause out of memory issues.
          model_loading_timeout (int): The worker model loading timeout in seconds. Defaults to
              None. If not provided, the default is 240 seconds.
          prediction_timeout (int): The worker predict call (handler) timeout in seconds.
              Defaults to None. If not provided, the default is 120 seconds.
          entry_point (str): This can either be the absolute or relative path to the Python source
              file that should be executed as the entry point to model
              hosting, or a python module that is installed in the container. If ``source_dir``
              is specified, then ``entry_point``
              must point to a file located at the root of ``source_dir``. Defaults to None.
          image_uri (str): A docker image URI. Defaults to None. If not specified, a default
              image for DJL Serving will be used based on ``djl_version``. If ``djl_version``
              is not specified, the latest available container version will be used.
          predictor_cls (callable[str, sagemaker.session.Session]): A function to call to create a
              predictor with an endpoint name and SageMaker ``Session``. If specified,
              ``deploy()`` returns
              the result of invoking this function on the created endpoint name.
          **kwargs: Keyword arguments passed to the superclass
              :class:`~sagemaker.model.FrameworkModel` and, subsequently, its
              superclass :class:`~sagemaker.model.Model`.
  
      .. tip::
  
          Instantiating a DJLModel will return an instance of either
          :class:`~sagemaker.djl_inference.DeepSpeedModel` or
          :class:`~sagemaker.djl_inference.HuggingFaceAccelerateModel` based on our framework
          recommendation for the model type.
  
          If you want to use a specific framework to deploy your model with, we recommend
          instantiating that specific
          model class directly. The available framework specific classes are
          :class:`~sagemaker.djl_inference.DeepSpeedModel` or
          :class:`~sagemaker.djl_inference.HuggingFaceAccelerateModel`
      """
      if "hf_hub_token" in kwargs:
          kwargs.pop("hf_hub_token")
      if kwargs.get("model_data"):
          logger.warning(
              "DJLModels do not use model_data parameter. model_data parameter will be ignored."
              "You only need to set model_id and ensure it points to uncompressed model "
              "artifacts in s3, or a valid HuggingFace Hub model_id."
          )
      data_type = kwargs.pop("data_type", None)
      if data_type:
          logger.warning(
              "data_type is being deprecated in favor of dtype. Please migrate use of data_type"
              " to dtype. Support for data_type will be removed in a future release"
          )
          dtype = dtype or data_type
      super(DJLModel, self).__init__(
          None,
          image_uri,
          role,
          entry_point,
          predictor_cls=predictor_cls,
          **kwargs,
      )
      self.model_id = model_id
      self.djl_version = djl_version
      self.task = task
      self.dtype = dtype
      self.number_of_partitions = number_of_partitions
      self.min_workers = min_workers
      self.max_workers = max_workers
      self.job_queue_size = job_queue_size
      self.parallel_loading = parallel_loading
      self.model_loading_timeout = model_loading_timeout
      self.prediction_timeout = prediction_timeout
      self.sagemaker_session = self.sagemaker_session or Session()
      self.save_mp_checkpoint_path = None
  
imports: from __future__ import absolute_import
  import json
  import logging
  import os.path
  import urllib.request
  from json import JSONDecodeError
  from urllib.error import HTTPError, URLError
  from enum import Enum
  from typing import Optional, Union, Dict, Any, List
  import sagemaker
  from sagemaker import s3, Predictor, image_uris, fw_utils
  from sagemaker.deserializers import JSONDeserializer, BaseDeserializer
  from sagemaker.djl_inference import defaults
  from sagemaker.model import FrameworkModel
  from sagemaker.s3_utils import s3_path_join
  from sagemaker.serializers import JSONSerializer, BaseSerializer
  from sagemaker.session import Session
  from sagemaker.utils import _tmpdir, _create_or_update_code_dir, format_tags
  from sagemaker.workflow.entities import PipelineVariable
  from sagemaker.estimator import Estimator
  from sagemaker.s3 import S3Uploader
label_desc: Use image node_id 2 to create a new DJLModel object from inherited base classes: FrameworkModel with example: obj = DJLModel(model_id, role, djl_version, task, dtype, number_of_partitions, min_workers, max_workers, job_queue_size, parallel_loading, model_loading_timeout, prediction_timeout, entry_point, image_uri, predictor_cls)