class_docstr: Run Python functions asynchronously as SageMaker jobs
class_docstr_tok: ["Run","Python","functions","asynchronously","as","SageMaker","jobs"]
docstr: Constructor for RemoteExecutor
  
  If a parameter value is not set, the constructor first looks up the value from the
  SageMaker configuration file. If no value is specified in the configuration file or
  no configuration file is found, the constructor selects the default as specified below.
  For more information, see `Configuring and using defaults with the SageMaker Python SDK
  <https://sagemaker.readthedocs.io/en/stable/overview.html
  #configuring-and-using-defaults-with-the-sagemaker-python-sdk>`_.
  
  Args:
      _func (Optional): A Python function to run as a SageMaker training job.
  
      dependencies (str): Either the path to a dependencies file or the reserved keyword
        ``auto_capture``. Defaults to ``None``.
        If ``dependencies`` is provided, the value must be one of the following:
  
      * A path to a conda environment.yml file. The following conditions apply.
  
        * If job_conda_env is set, then the conda environment is updated by installing
          dependencies from the yaml file and the function is invoked within that
          conda environment. For this to succeed, the specified conda environment must
          already exist in the image.
        * If the environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image, then
          the conda environment is updated by installing dependencies from the yaml file and
          the function is invoked within that conda environment. For this to succeed, the
          conda environment name must already be set in ``SAGEMAKER_JOB_CONDA_ENV``, and
          ``SAGEMAKER_JOB_CONDA_ENV`` must already exist in the image.
        * If none of the previous conditions are met, a new conda environment named
          ``sagemaker-runtime-env`` is created and the function annotated with the remote
          decorator is invoked in that conda environment.
  
      * A path to a requirements.txt file. The following conditions apply.
  
        * If ``job_conda_env`` is set in the remote decorator, dependencies are installed
          within that conda environment and the function annotated with the remote decorator
          is invoked in the same conda environment. For this to succeed, the specified
          conda environment must already exist in the image.
        * If an environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image,
          dependencies are installed within that conda environment and the function annotated
          with the remote decorator is invoked in the same. For this to succeed, the
          conda environment name must already be set in ``SAGEMAKER_JOB_CONDA_ENV``, and
          ``SAGEMAKER_JOB_CONDA_ENV`` must already exist in the image.
        * If none of the above conditions are met, conda is not used. Dependencies are
          installed at the system level, without any virtual environment, and the function
          annotated with the remote decorator is invoked using the Python runtime available
          in the system path.
  
      * The parameter dependencies is set to ``auto_capture``. SageMaker will automatically
          generate an env_snapshot.yml corresponding to the current active conda environment’s
          snapshot. You do not need to provide a dependencies file. The following conditions
          apply:
  
        * You must run the remote function within an active conda environment.
        * When installing the dependencies on the training job, the same conditions as when
          dependencies is set to a path to a conda environment file apply. These conditions
          are as follows:
  
            * If job_conda_env is set, then the conda environment is updated by installing
              dependencies from the yaml file and the function is invoked within that
              conda environment. For this to succeed, the specified conda environment must
              already exist in the image.
            * If the environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image,
              then the conda environment is updated by installing dependencies from the yaml
              file and the function is invoked within that conda environment. For this to
              succeed, the conda environment name must already be set in
              ``SAGEMAKER_JOB_CONDA_ENV``, and ``SAGEMAKER_JOB_CONDA_ENV`` must already exist
              in the image.
            * If none of the previous conditions are met, a new conda environment with name
              ``sagemaker-runtime-env`` is created and the function annotated with the
              remote decorator is invoked in that conda environment.
  
        * ``None``. SageMaker will assume that there are no dependencies to install while
          executing the remote annotated function in the training job.
  
      pre_execution_commands (List[str]): List of commands to be executed prior to executing
        remote function. Only one of ``pre_execution_commands`` or ``pre_execution_script``
        can be specified at the same time. Defaults to None.
  
      pre_execution_script (str): Path to script file to be executed prior to executing
        remote function. Only one of ``pre_execution_commands`` or ``pre_execution_script``
        can be specified at the same time. Defaults to None.
  
      environment_variables (Dict): The environment variables used inside the decorator
        function. Defaults to ``None``.
  
      image_uri (str): The universal resource identifier (URI) location of a Docker image on
        Amazon Elastic Container Registry (ECR). Defaults to the following based on where the
        SDK is running:
  
        * For users who specify ``spark_config`` and want to run the function in a Spark
          application, the ``image_uri`` should be ``None``. A SageMaker Spark image will
          be used for training, otherwise a ``ValueError`` is thrown.
        * For users on SageMaker Studio notebooks, the image used as the kernel image for
          the notebook is used.
        * For other users, it is resolved to base python image with the same python
          version as the environment running the local code.
  
        If no compatible image is found, a ValueError is thrown.
  
      include_local_workdir (bool): A flag to indicate that the remote function should include
        local directories. Set to ``True`` if the remote function code imports local modules
        and methods that are not available via PyPI or conda. Default value is ``False``.
  
      custom_file_filter (Callable[[str, List], List], CustomFileFilter): Either a function
        that filters job dependencies to be uploaded to S3 or a ``CustomFileFilter`` object
        that specifies the local directories and files to be included in the remote function.
        If a callable is passed in, that function is passed to the ``ignore``  argument of
        ``shutil.copytree``. Defaults to ``None``, which means only python
        files are accepted and uploaded to S3.
  
      instance_count (int): The number of instances to use. Defaults to 1.
        NOTE: Remote function does not support instance_count > 1 for non Spark jobs.
  
      instance_type (str): The Amazon Elastic Compute Cloud (EC2) instance type to use to run
        the SageMaker job. e.g. ml.c4.xlarge. If not provided, a ValueError is thrown.
  
      job_conda_env (str): The name of the conda environment to activate during job's runtime.
        Defaults to ``None``.
  
      job_name_prefix (str): The prefix used used to create the underlying SageMaker job.
  
      keep_alive_period_in_seconds (int): The duration in seconds to retain and reuse
        provisioned infrastructure after the completion of a training job, also known as
        SageMaker managed warm pools. The use of warmpools reduces the latency time spent to
        provision new resources. The default value for ``keep_alive_period_in_seconds`` is 0.
        NOTE: Additional charges associated with warm pools may apply. Using this parameter
        also activates a new pesistent cache feature, which will further reduce job start
        up latency than over using SageMaker managed warm pools alone by caching the package
        source downloaded in the previous runs.
  
      max_parallel_jobs (int): Maximum number of jobs that run in parallel. Defaults to 1.
  
      max_retry_attempts (int): The max number of times the job is retried on
        ``InternalServerFailure`` Error from SageMaker service. Defaults to 1.
  
      max_runtime_in_seconds (int): The upper limit in seconds to be used for training. After
        this specified amount of time, SageMaker terminates the job regardless of its current
        status. Defaults to 1 day or (86400 seconds).
  
      role (str): The IAM role (either name or full ARN) used to run your SageMaker training
        job. Defaults to:
  
        * the SageMaker default IAM role if the SDK is running in SageMaker Notebooks or
          SageMaker Studio Notebooks.
        * if not above, a ValueError is be thrown.
  
      s3_kms_key (str): The key used to encrypt the input and output data.
        Default to ``None``.
  
      s3_root_uri (str): The root S3 folder to which the code archives and data are
        uploaded to. Defaults to ``s3://<sagemaker-default-bucket>``.
  
      sagemaker_session (sagemaker.session.Session): The underlying SageMaker session to which
        SageMaker service calls are delegated to (default: None). If not provided, one is
        created using a default configuration chain.
  
      security_group_ids (List[str): A list of security group IDs. Defaults to ``None`` and
        the training job is created without VPC config.
  
      subnets (List[str): A list of subnet IDs. Defaults to ``None`` and the job is
        created without VPC config.
  
      tags (List[Tuple[str, str]): A list of tags attached to the job. Defaults to ``None``
        and the training job is created without tags.
  
      volume_kms_key (str): An Amazon Key Management Service (KMS) key used to encrypt an
        Amazon Elastic Block Storage (EBS) volume attached to the training instance.
        Defaults to ``None``.
  
      volume_size (int): The size in GB of the storage volume for storing input and output
        data during training. Defaults to ``30``.
  
      encrypt_inter_container_traffic (bool): A flag that specifies whether traffic between
        training containers is encrypted for the training job. Defaults to ``False``.
  
      enable_network_isolation (bool): A flag that specifies whether container will run in
        network isolation mode. Defaults to ``False``. Network isolation mode restricts the
        container access to outside networks (such as the Internet). The container does not
        make any inbound or outbound network calls. Also known as Internet-free mode.
  
      spark_config (SparkConfig): Configurations to the Spark application that runs on
        Spark image. If ``spark_config`` is specified, a SageMaker Spark image uri
        will be used for training. Note that ``image_uri`` can not be specified at the
        same time otherwise a ``ValueError`` is thrown. Defaults to ``None``.
  
      use_spot_instances (bool): Specifies whether to use SageMaker Managed Spot instances for
        training. If enabled then the ``max_wait_time_in_seconds`` arg should also be set.
        Defaults to ``False``.
  
      max_wait_time_in_seconds (int): Timeout in seconds waiting for spot training job.
        After this amount of time Amazon SageMaker will stop waiting for managed spot training
        job to complete. Defaults to ``None``.
docstr_tok: ["Constructor","for","RemoteExecutor","If","a","parameter","value","is","not","set",",","the","constructor","first","looks","up","the","value","from","the","SageMaker","configuration","file",".","If","no","value","is","specified","in","the","configuration","file","or","no","configuration","file","is","found",",","the","constructor","selects","the","default","as","specified","below",".","For","more","information",",","see","`","Configuring","and","using","defaults","with","the","SageMaker","Python","SDK","<","https",":","\/\/sagemaker.readthedocs.io\/en\/stable\/overview.html","#","configuring-and-using-defaults-with-the-sagemaker-python-sdk",">","`","_",".","Args",":","_func","(","Optional",")",":","A","Python","function","to","run","as","a","SageMaker","training","job",".","dependencies","(","str",")",":","Either","the","path","to","a","dependencies","file","or","the","reserved","keyword","``","auto_capture","``",".","Defaults","to","``","None","``",".","If","``","dependencies","``","is","provided",",","the","value","must","be","one","of","the","following",":","*","A","path","to","a","conda","environment.yml","file",".","The","following","conditions","apply",".","*","If","job_conda_env","is","set",",","then","the","conda","environment","is","updated","by","installing","dependencies","from","the","yaml","file","and","the","function","is","invoked","within","that","conda","environment",".","For","this","to","succeed",",","the","specified","conda","environment","must","already","exist","in","the","image",".","*","If","the","environment","variable","``","SAGEMAKER_JOB_CONDA_ENV","``","is","set","in","the","image",",","then","the","conda","environment","is","updated","by","installing","dependencies","from","the","yaml","file","and","the","function","is","invoked","within","that","conda","environment",".","For","this","to","succeed",",","the","conda","environment","name","must","already","be","set","in","``","SAGEMAKER_JOB_CONDA_ENV","``",",","and","``","SAGEMAKER_JOB_CONDA_ENV","``","must","already","exist","in","the","image",".","*","If","none","of","the","previous","conditions","are","met",",","a","new","conda","environment","named","``","sagemaker-runtime-env","``","is","created","and","the","function","annotated","with","the","remote","decorator","is","invoked","in","that","conda","environment",".","*","A","path","to","a","requirements.txt","file",".","The","following","conditions","apply",".","*","If","``","job_conda_env","``","is","set","in","the","remote","decorator",",","dependencies","are","installed","within","that","conda","environment","and","the","function","annotated","with","the","remote","decorator","is","invoked","in","the","same","conda","environment",".","For","this","to","succeed",",","the","specified","conda","environment","must","already","exist","in","the","image",".","*","If","an","environment","variable","``","SAGEMAKER_JOB_CONDA_ENV","``","is","set","in","the","image",",","dependencies","are","installed","within","that","conda","environment","and","the","function","annotated","with","the","remote","decorator","is","invoked","in","the","same",".","For","this","to","succeed",",","the","conda","environment","name","must","already","be","set","in","``","SAGEMAKER_JOB_CONDA_ENV","``",",","and","``","SAGEMAKER_JOB_CONDA_ENV","``","must","already","exist","in","the","image",".","*","If","none","of","the","above","conditions","are","met",",","conda","is","not","used",".","Dependencies","are","installed","at","the","system","level",",","without","any","virtual","environment",",","and","the","function","annotated","with","the","remote","decorator","is","invoked","using","the","Python","runtime","available","in","the","system","path",".","*","The","parameter","dependencies","is","set","to","``","auto_capture","``",".","SageMaker","will","automatically","generate","an","env_snapshot.yml","corresponding","to","the","current","active","conda","environment","\u2019","s","snapshot",".","You","do","not","need","to","provide","a","dependencies","file",".","The","following","conditions","apply",":","*","You","must","run","the","remote","function","within","an","active","conda","environment",".","*","When","installing","the","dependencies","on","the","training","job",",","the","same","conditions","as","when","dependencies","is","set","to","a","path","to","a","conda","environment","file","apply",".","These","conditions","are","as","follows",":","*","If","job_conda_env","is","set",",","then","the","conda","environment","is","updated","by","installing","dependencies","from","the","yaml","file","and","the","function","is","invoked","within","that","conda","environment",".","For","this","to","succeed",",","the","specified","conda","environment","must","already","exist","in","the","image",".","*","If","the","environment","variable","``","SAGEMAKER_JOB_CONDA_ENV","``","is","set","in","the","image",",","then","the","conda","environment","is","updated","by","installing","dependencies","from","the","yaml","file","and","the","function","is","invoked","within","that","conda","environment",".","For","this","to","succeed",",","the","conda","environment","name","must","already","be","set","in","``","SAGEMAKER_JOB_CONDA_ENV","``",",","and","``","SAGEMAKER_JOB_CONDA_ENV","``","must","already","exist","in","the","image",".","*","If","none","of","the","previous","conditions","are","met",",","a","new","conda","environment","with","name","``","sagemaker-runtime-env","``","is","created","and","the","function","annotated","with","the","remote","decorator","is","invoked","in","that","conda","environment",".","*","``","None","``",".","SageMaker","will","assume","that","there","are","no","dependencies","to","install","while","executing","the","remote","annotated","function","in","the","training","job",".","pre_execution_commands","(","List","[","str","]",")",":","List","of","commands","to","be","executed","prior","to","executing","remote","function",".","Only","one","of","``","pre_execution_commands","``","or","``","pre_execution_script","``","can","be","specified","at","the","same","time",".","Defaults","to","None",".","pre_execution_script","(","str",")",":","Path","to","script","file","to","be","executed","prior","to","executing","remote","function",".","Only","one","of","``","pre_execution_commands","``","or","``","pre_execution_script","``","can","be","specified","at","the","same","time",".","Defaults","to","None",".","environment_variables","(","Dict",")",":","The","environment","variables","used","inside","the","decorator","function",".","Defaults","to","``","None","``",".","image_uri","(","str",")",":","The","universal","resource","identifier","(","URI",")","location","of","a","Docker","image","on","Amazon","Elastic","Container","Registry","(","ECR",")",".","Defaults","to","the","following","based","on","where","the","SDK","is","running",":","*","For","users","who","specify","``","spark_config","``","and","want","to","run","the","function","in","a","Spark","application",",","the","``","image_uri","``","should","be","``","None","``",".","A","SageMaker","Spark","image","will","be","used","for","training",",","otherwise","a","``","ValueError","``","is","thrown",".","*","For","users","on","SageMaker","Studio","notebooks",",","the","image","used","as","the","kernel","image","for","the","notebook","is","used",".","*","For","other","users",",","it","is","resolved","to","base","python","image","with","the","same","python","version","as","the","environment","running","the","local","code",".","If","no","compatible","image","is","found",",","a","ValueError","is","thrown",".","include_local_workdir","(","bool",")",":","A","flag","to","indicate","that","the","remote","function","should","include","local","directories",".","Set","to","``","True","``","if","the","remote","function","code","imports","local","modules","and","methods","that","are","not","available","via","PyPI","or","conda",".","Default","value","is","``","False","``",".","custom_file_filter","(","Callable","[","[","str",",","List","]",",","List","]",",","CustomFileFilter",")",":","Either","a","function","that","filters","job","dependencies","to","be","uploaded","to","S3","or","a","``","CustomFileFilter","``","object","that","specifies","the","local","directories","and","files","to","be","included","in","the","remote","function",".","If","a","callable","is","passed","in",",","that","function","is","passed","to","the","``","ignore","``","argument","of","``","shutil.copytree","``",".","Defaults","to","``","None","``",",","which","means","only","python","files","are","accepted","and","uploaded","to","S3",".","instance_count","(","int",")",":","The","number","of","instances","to","use",".","Defaults","to","1",".","NOTE",":","Remote","function","does","not","support","instance_count",">","1","for","non","Spark","jobs",".","instance_type","(","str",")",":","The","Amazon","Elastic","Compute","Cloud","(","EC2",")","instance","type","to","use","to","run","the","SageMaker","job",".","e.g",".","ml.c4.xlarge",".","If","not","provided",",","a","ValueError","is","thrown",".","job_conda_env","(","str",")",":","The","name","of","the","conda","environment","to","activate","during","job","'s","runtime",".","Defaults","to","``","None","``",".","job_name_prefix","(","str",")",":","The","prefix","used","used","to","create","the","underlying","SageMaker","job",".","keep_alive_period_in_seconds","(","int",")",":","The","duration","in","seconds","to","retain","and","reuse","provisioned","infrastructure","after","the","completion","of","a","training","job",",","also","known","as","SageMaker","managed","warm","pools",".","The","use","of","warmpools","reduces","the","latency","time","spent","to","provision","new","resources",".","The","default","value","for","``","keep_alive_period_in_seconds","``","is","0",".","NOTE",":","Additional","charges","associated","with","warm","pools","may","apply",".","Using","this","parameter","also","activates","a","new","pesistent","cache","feature",",","which","will","further","reduce","job","start","up","latency","than","over","using","SageMaker","managed","warm","pools","alone","by","caching","the","package","source","downloaded","in","the","previous","runs",".","max_parallel_jobs","(","int",")",":","Maximum","number","of","jobs","that","run","in","parallel",".","Defaults","to","1.","max_retry_attempts","(","int",")",":","The","max","number","of","times","the","job","is","retried","on","``","InternalServerFailure","``","Error","from","SageMaker","service",".","Defaults","to","1.","max_runtime_in_seconds","(","int",")",":","The","upper","limit","in","seconds","to","be","used","for","training",".","After","this","specified","amount","of","time",",","SageMaker","terminates","the","job","regardless","of","its","current","status",".","Defaults","to","1","day","or","(","86400","seconds",")",".","role","(","str",")",":","The","IAM","role","(","either","name","or","full","ARN",")","used","to","run","your","SageMaker","training","job",".","Defaults","to",":","*","the","SageMaker","default","IAM","role","if","the","SDK","is","running","in","SageMaker","Notebooks","or","SageMaker","Studio","Notebooks",".","*","if","not","above",",","a","ValueError","is","be","thrown",".","s3_kms_key","(","str",")",":","The","key","used","to","encrypt","the","input","and","output","data",".","Default","to","``","None","``",".","s3_root_uri","(","str",")",":","The","root","S3","folder","to","which","the","code","archives","and","data","are","uploaded","to",".","Defaults","to","``","s3",":","\/\/","<","sagemaker-default-bucket",">","``",".","sagemaker_session","(","sagemaker.session.Session",")",":","The","underlying","SageMaker","session","to","which","SageMaker","service","calls","are","delegated","to","(","default",":","None",")",".","If","not","provided",",","one","is","created","using","a","default","configuration","chain",".","security_group_ids","(","List","[","str",")",":","A","list","of","security","group","IDs",".","Defaults","to","``","None","``","and","the","training","job","is","created","without","VPC","config",".","subnets","(","List","[","str",")",":","A","list","of","subnet","IDs",".","Defaults","to","``","None","``","and","the","job","is","created","without","VPC","config",".","tags","(","List","[","Tuple","[","str",",","str","]",")",":","A","list","of","tags","attached","to","the","job",".","Defaults","to","``","None","``","and","the","training","job","is","created","without","tags",".","volume_kms_key","(","str",")",":","An","Amazon","Key","Management","Service","(","KMS",")","key","used","to","encrypt","an","Amazon","Elastic","Block","Storage","(","EBS",")","volume","attached","to","the","training","instance",".","Defaults","to","``","None","``",".","volume_size","(","int",")",":","The","size","in","GB","of","the","storage","volume","for","storing","input","and","output","data","during","training",".","Defaults","to","``","30","``",".","encrypt_inter_container_traffic","(","bool",")",":","A","flag","that","specifies","whether","traffic","between","training","containers","is","encrypted","for","the","training","job",".","Defaults","to","``","False","``",".","enable_network_isolation","(","bool",")",":","A","flag","that","specifies","whether","container","will","run","in","network","isolation","mode",".","Defaults","to","``","False","``",".","Network","isolation","mode","restricts","the","container","access","to","outside","networks","(","such","as","the","Internet",")",".","The","container","does","not","make","any","inbound","or","outbound","network","calls",".","Also","known","as","Internet-free","mode",".","spark_config","(","SparkConfig",")",":","Configurations","to","the","Spark","application","that","runs","on","Spark","image",".","If","``","spark_config","``","is","specified",",","a","SageMaker","Spark","image","uri","will","be","used","for","training",".","Note","that","``","image_uri","``","can","not","be","specified","at","the","same","time","otherwise","a","``","ValueError","``","is","thrown",".","Defaults","to","``","None","``",".","use_spot_instances","(","bool",")",":","Specifies","whether","to","use","SageMaker","Managed","Spot","instances","for","training",".","If","enabled","then","the","``","max_wait_time_in_seconds","``","arg","should","also","be","set",".","Defaults","to","``","False","``",".","max_wait_time_in_seconds","(","int",")",":","Timeout","in","seconds","waiting","for","spot","training","job",".","After","this","amount","of","time","Amazon","SageMaker","will","stop","waiting","for","managed","spot","training","job","to","complete",".","Defaults","to","``","None","``","."]
code: def __init__(
      self,
      *,
      dependencies: str = None,
      pre_execution_commands: List[str] = None,
      pre_execution_script: str = None,
      environment_variables: Dict[str, str] = None,
      image_uri: str = None,
      include_local_workdir: bool = None,
      custom_file_filter: Optional[
          Union[Callable[[str, List], List], CustomFileFilter]
      ] = None,
      instance_count: int = 1,
      instance_type: str = None,
      job_conda_env: str = None,
      job_name_prefix: str = None,
      keep_alive_period_in_seconds: int = 0,
      max_parallel_jobs: int = 1,
      max_retry_attempts: int = 1,
      max_runtime_in_seconds: int = 24 * 60 * 60,
      role: str = None,
      s3_kms_key: str = None,
      s3_root_uri: str = None,
      sagemaker_session: Session = None,
      security_group_ids: List[str] = None,
      subnets: List[str] = None,
      tags: List[Tuple[str, str]] = None,
      volume_kms_key: str = None,
      volume_size: int = 30,
      encrypt_inter_container_traffic: bool = None,
      spark_config: SparkConfig = None,
      use_spot_instances=False,
      max_wait_time_in_seconds=None,
  ):
      """Constructor for RemoteExecutor
  
      If a parameter value is not set, the constructor first looks up the value from the
      SageMaker configuration file. If no value is specified in the configuration file or
      no configuration file is found, the constructor selects the default as specified below.
      For more information, see `Configuring and using defaults with the SageMaker Python SDK
      <https://sagemaker.readthedocs.io/en/stable/overview.html
      #configuring-and-using-defaults-with-the-sagemaker-python-sdk>`_.
  
      Args:
          _func (Optional): A Python function to run as a SageMaker training job.
  
          dependencies (str): Either the path to a dependencies file or the reserved keyword
            ``auto_capture``. Defaults to ``None``.
            If ``dependencies`` is provided, the value must be one of the following:
  
          * A path to a conda environment.yml file. The following conditions apply.
  
            * If job_conda_env is set, then the conda environment is updated by installing
              dependencies from the yaml file and the function is invoked within that
              conda environment. For this to succeed, the specified conda environment must
              already exist in the image.
            * If the environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image, then
              the conda environment is updated by installing dependencies from the yaml file and
              the function is invoked within that conda environment. For this to succeed, the
              conda environment name must already be set in ``SAGEMAKER_JOB_CONDA_ENV``, and
              ``SAGEMAKER_JOB_CONDA_ENV`` must already exist in the image.
            * If none of the previous conditions are met, a new conda environment named
              ``sagemaker-runtime-env`` is created and the function annotated with the remote
              decorator is invoked in that conda environment.
  
          * A path to a requirements.txt file. The following conditions apply.
  
            * If ``job_conda_env`` is set in the remote decorator, dependencies are installed
              within that conda environment and the function annotated with the remote decorator
              is invoked in the same conda environment. For this to succeed, the specified
              conda environment must already exist in the image.
            * If an environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image,
              dependencies are installed within that conda environment and the function annotated
              with the remote decorator is invoked in the same. For this to succeed, the
              conda environment name must already be set in ``SAGEMAKER_JOB_CONDA_ENV``, and
              ``SAGEMAKER_JOB_CONDA_ENV`` must already exist in the image.
            * If none of the above conditions are met, conda is not used. Dependencies are
              installed at the system level, without any virtual environment, and the function
              annotated with the remote decorator is invoked using the Python runtime available
              in the system path.
  
          * The parameter dependencies is set to ``auto_capture``. SageMaker will automatically
              generate an env_snapshot.yml corresponding to the current active conda environment’s
              snapshot. You do not need to provide a dependencies file. The following conditions
              apply:
  
            * You must run the remote function within an active conda environment.
            * When installing the dependencies on the training job, the same conditions as when
              dependencies is set to a path to a conda environment file apply. These conditions
              are as follows:
  
                * If job_conda_env is set, then the conda environment is updated by installing
                  dependencies from the yaml file and the function is invoked within that
                  conda environment. For this to succeed, the specified conda environment must
                  already exist in the image.
                * If the environment variable ``SAGEMAKER_JOB_CONDA_ENV`` is set in the image,
                  then the conda environment is updated by installing dependencies from the yaml
                  file and the function is invoked within that conda environment. For this to
                  succeed, the conda environment name must already be set in
                  ``SAGEMAKER_JOB_CONDA_ENV``, and ``SAGEMAKER_JOB_CONDA_ENV`` must already exist
                  in the image.
                * If none of the previous conditions are met, a new conda environment with name
                  ``sagemaker-runtime-env`` is created and the function annotated with the
                  remote decorator is invoked in that conda environment.
  
            * ``None``. SageMaker will assume that there are no dependencies to install while
              executing the remote annotated function in the training job.
  
          pre_execution_commands (List[str]): List of commands to be executed prior to executing
            remote function. Only one of ``pre_execution_commands`` or ``pre_execution_script``
            can be specified at the same time. Defaults to None.
  
          pre_execution_script (str): Path to script file to be executed prior to executing
            remote function. Only one of ``pre_execution_commands`` or ``pre_execution_script``
            can be specified at the same time. Defaults to None.
  
          environment_variables (Dict): The environment variables used inside the decorator
            function. Defaults to ``None``.
  
          image_uri (str): The universal resource identifier (URI) location of a Docker image on
            Amazon Elastic Container Registry (ECR). Defaults to the following based on where the
            SDK is running:
  
            * For users who specify ``spark_config`` and want to run the function in a Spark
              application, the ``image_uri`` should be ``None``. A SageMaker Spark image will
              be used for training, otherwise a ``ValueError`` is thrown.
            * For users on SageMaker Studio notebooks, the image used as the kernel image for
              the notebook is used.
            * For other users, it is resolved to base python image with the same python
              version as the environment running the local code.
  
            If no compatible image is found, a ValueError is thrown.
  
          include_local_workdir (bool): A flag to indicate that the remote function should include
            local directories. Set to ``True`` if the remote function code imports local modules
            and methods that are not available via PyPI or conda. Default value is ``False``.
  
          custom_file_filter (Callable[[str, List], List], CustomFileFilter): Either a function
            that filters job dependencies to be uploaded to S3 or a ``CustomFileFilter`` object
            that specifies the local directories and files to be included in the remote function.
            If a callable is passed in, that function is passed to the ``ignore``  argument of
            ``shutil.copytree``. Defaults to ``None``, which means only python
            files are accepted and uploaded to S3.
  
          instance_count (int): The number of instances to use. Defaults to 1.
            NOTE: Remote function does not support instance_count > 1 for non Spark jobs.
  
          instance_type (str): The Amazon Elastic Compute Cloud (EC2) instance type to use to run
            the SageMaker job. e.g. ml.c4.xlarge. If not provided, a ValueError is thrown.
  
          job_conda_env (str): The name of the conda environment to activate during job's runtime.
            Defaults to ``None``.
  
          job_name_prefix (str): The prefix used used to create the underlying SageMaker job.
  
          keep_alive_period_in_seconds (int): The duration in seconds to retain and reuse
            provisioned infrastructure after the completion of a training job, also known as
            SageMaker managed warm pools. The use of warmpools reduces the latency time spent to
            provision new resources. The default value for ``keep_alive_period_in_seconds`` is 0.
            NOTE: Additional charges associated with warm pools may apply. Using this parameter
            also activates a new pesistent cache feature, which will further reduce job start
            up latency than over using SageMaker managed warm pools alone by caching the package
            source downloaded in the previous runs.
  
          max_parallel_jobs (int): Maximum number of jobs that run in parallel. Defaults to 1.
  
          max_retry_attempts (int): The max number of times the job is retried on
            ``InternalServerFailure`` Error from SageMaker service. Defaults to 1.
  
          max_runtime_in_seconds (int): The upper limit in seconds to be used for training. After
            this specified amount of time, SageMaker terminates the job regardless of its current
            status. Defaults to 1 day or (86400 seconds).
  
          role (str): The IAM role (either name or full ARN) used to run your SageMaker training
            job. Defaults to:
  
            * the SageMaker default IAM role if the SDK is running in SageMaker Notebooks or
              SageMaker Studio Notebooks.
            * if not above, a ValueError is be thrown.
  
          s3_kms_key (str): The key used to encrypt the input and output data.
            Default to ``None``.
  
          s3_root_uri (str): The root S3 folder to which the code archives and data are
            uploaded to. Defaults to ``s3://<sagemaker-default-bucket>``.
  
          sagemaker_session (sagemaker.session.Session): The underlying SageMaker session to which
            SageMaker service calls are delegated to (default: None). If not provided, one is
            created using a default configuration chain.
  
          security_group_ids (List[str): A list of security group IDs. Defaults to ``None`` and
            the training job is created without VPC config.
  
          subnets (List[str): A list of subnet IDs. Defaults to ``None`` and the job is
            created without VPC config.
  
          tags (List[Tuple[str, str]): A list of tags attached to the job. Defaults to ``None``
            and the training job is created without tags.
  
          volume_kms_key (str): An Amazon Key Management Service (KMS) key used to encrypt an
            Amazon Elastic Block Storage (EBS) volume attached to the training instance.
            Defaults to ``None``.
  
          volume_size (int): The size in GB of the storage volume for storing input and output
            data during training. Defaults to ``30``.
  
          encrypt_inter_container_traffic (bool): A flag that specifies whether traffic between
            training containers is encrypted for the training job. Defaults to ``False``.
  
          enable_network_isolation (bool): A flag that specifies whether container will run in
            network isolation mode. Defaults to ``False``. Network isolation mode restricts the
            container access to outside networks (such as the Internet). The container does not
            make any inbound or outbound network calls. Also known as Internet-free mode.
  
          spark_config (SparkConfig): Configurations to the Spark application that runs on
            Spark image. If ``spark_config`` is specified, a SageMaker Spark image uri
            will be used for training. Note that ``image_uri`` can not be specified at the
            same time otherwise a ``ValueError`` is thrown. Defaults to ``None``.
  
          use_spot_instances (bool): Specifies whether to use SageMaker Managed Spot instances for
            training. If enabled then the ``max_wait_time_in_seconds`` arg should also be set.
            Defaults to ``False``.
  
          max_wait_time_in_seconds (int): Timeout in seconds waiting for spot training job.
            After this amount of time Amazon SageMaker will stop waiting for managed spot training
            job to complete. Defaults to ``None``.
      """
      self.max_parallel_jobs = max_parallel_jobs
  
      if self.max_parallel_jobs <= 0:
          raise ValueError("max_parallel_jobs must be greater than 0.")
  
      if instance_count > 1 and not spark_config:
          raise ValueError(
              "Remote function do not support training on multi instances. "
              + "Please provide instance_count = 1"
          )
  
      self.job_settings = _JobSettings(
          dependencies=dependencies,
          pre_execution_commands=pre_execution_commands,
          pre_execution_script=pre_execution_script,
          environment_variables=environment_variables,
          image_uri=image_uri,
          include_local_workdir=include_local_workdir,
          custom_file_filter=custom_file_filter,
          instance_count=instance_count,
          instance_type=instance_type,
          job_conda_env=job_conda_env,
          job_name_prefix=job_name_prefix,
          keep_alive_period_in_seconds=keep_alive_period_in_seconds,
          max_retry_attempts=max_retry_attempts,
          max_runtime_in_seconds=max_runtime_in_seconds,
          role=role,
          s3_kms_key=s3_kms_key,
          s3_root_uri=s3_root_uri,
          sagemaker_session=sagemaker_session,
          security_group_ids=security_group_ids,
          subnets=subnets,
          tags=tags,
          volume_kms_key=volume_kms_key,
          volume_size=volume_size,
          encrypt_inter_container_traffic=encrypt_inter_container_traffic,
          spark_config=spark_config,
          use_spot_instances=use_spot_instances,
          max_wait_time_in_seconds=max_wait_time_in_seconds,
      )
  
      self._state_condition = threading.Condition()
      self._pending_request_queue = deque()
      # For thread safety, see
      # https://web.archive.org/web/20201108091210/http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm
      self._running_jobs = dict()
      self._shutdown = False
  
      self._workers: ThreadPoolExecutor = None
  
imports: from __future__ import absolute_import
  from concurrent.futures import ThreadPoolExecutor
  from collections import deque
  import time
  import threading
  from typing import Callable, Dict, List, Optional, Tuple, Any, Union
  import functools
  import itertools
  import inspect
  from botocore.exceptions import ClientError
  from sagemaker.exceptions import UnexpectedStatusException
  from sagemaker.experiments._run_context import _RunContext
  import sagemaker.remote_function.core.serialization
  from sagemaker.remote_function.errors import RemoteFunctionError, ServiceError, DeserializationError
  from sagemaker.remote_function.core.stored_function import RESULTS_FOLDER, EXCEPTION_FOLDER
  from sagemaker.remote_function.runtime_environment.runtime_environment_manager import RuntimeEnvironmentError
  from sagemaker.session import Session
  from sagemaker.s3 import s3_path_join
  from sagemaker.remote_function.job import _JobSettings, _Job, _RunInfo
  from sagemaker.remote_function import logging_config
  from sagemaker.utils import name_from_base, base_from_name
  from sagemaker.remote_function.spark_config import SparkConfig
  from sagemaker.remote_function.custom_file_filter import CustomFileFilter
label_desc: Use image node_id 1 to create a new RemoteExecutor object from inherited base classes: object with example: obj = RemoteExecutor()