docstr: Create a Task from a function, including wrapping the function input arguments
  into the hyper-parameter section as kwargs, and storing function results as named artifacts
  
  Example:
      def mock_func(a=6, b=9):
          c = a*b
          print(a, b, c)
          return c, c**2
  
      create_task_from_function(mock_func, function_return=['mul', 'square'])
  
  Example arguments from other Tasks (artifact):
      def mock_func(matrix_np):
          c = matrix_np*matrix_np
          print(matrix_np, c)
          return c
  
      create_task_from_function(
          mock_func,
          function_input_artifacts={'matrix_np': 'aabb1122.previous_matrix'},
          function_return=['square_matrix']
      )
  
  :param a_function: A global function to convert into a standalone Task
  :param function_kwargs: Optional, provide subset of function arguments and default values to expose.
      If not provided automatically take all function arguments & defaults
  :param function_input_artifacts: Optional, pass input arguments to the function from
      other Tasks's output artifact.
      Example argument named `numpy_matrix` from Task ID `aabbcc` artifact name `answer`:
      {'numpy_matrix': 'aabbcc.answer'}
  :param function_return: Provide a list of names for all the results.
      If not provided no results will be stored as artifacts.
  :param project_name: Set the project name for the task. Required if base_task_id is None.
  :param task_name: Set the name of the remote task. Required if base_task_id is None.
  :param task_type: Optional, The task type to be created. Supported values: 'training', 'testing', 'inference',
      'data_processing', 'application', 'monitor', 'controller', 'optimizer', 'service', 'qc', 'custom'
  :param auto_connect_frameworks: Control the frameworks auto connect, see `Task.init` auto_connect_frameworks
  :param auto_connect_arg_parser: Control the ArgParser auto connect, see `Task.init` auto_connect_arg_parser
  :param repo: Remote URL for the repository to use, OR path to local copy of the git repository
      Example: 'https://github.com/allegroai/clearml.git' or '~/project/repo'
  :param branch: Select specific repository branch/tag (implies the latest commit from the branch)
  :param commit: Select specific commit id to use (default: latest commit,
      or when used with local repository matching the local commit id)
  :param packages: Manually specify a list of required packages or a local requirements.txt file.
      Example: ["tqdm>=2.1", "scikit-learn"] or "./requirements.txt"
      If not provided, packages are automatically added based on the imports used in the function.
  :param docker: Select the docker image to be executed in by the remote session
  :param docker_args: Add docker arguments, pass a single string
  :param docker_bash_setup_script: Add bash script to be executed
      inside the docker before setting up the Task's environment
  :param output_uri: Optional, set the Tasks's output_uri (Storage destination).
      examples: 's3://bucket/folder', 'https://server/' , 'gs://bucket/folder', 'azure://bucket', '/folder/'
  :param helper_functions: Optional, a list of helper functions to make available
      for the standalone function Task.
  :param dry_run: If True, do not create the Task, but return a dict of the Task's definitions
  :param task_template_header: A string placed at the top of the task's code
  :param artifact_serialization_function: A serialization function that takes one
      parameter of any type which is the object to be serialized. The function should return
      a `bytes` or `bytearray` object, which represents the serialized object. All parameter/return
      artifacts uploaded by the pipeline will be serialized using this function.
      All relevant imports must be done in this function. For example:
  
      .. code-block:: py
  
          def serialize(obj):
              import dill
              return dill.dumps(obj)
  :param artifact_deserialization_function: A deserialization function that takes one parameter of type `bytes`,
      which represents the serialized object. This function should return the deserialized object.
      All parameter/return artifacts fetched by the pipeline will be deserialized using this function.
      All relevant imports must be done in this function. For example:
  
      .. code-block:: py
  
          def deserialize(bytes_):
              import dill
              return dill.loads(bytes_)
  :param _sanitize_function: Sanitization function for the function string.
  :param _sanitize_helper_functions: Sanitization function for the helper function string.
  :param skip_global_imports: If True, the global imports will not be fetched from the function's file, otherwise
      all global imports will be automatically imported in a safe manner at the beginning of the function's
      execution. Default is False
  :return: Newly created Task object
docstr_tok: ["Create","a","Task","from","a","function",",","including","wrapping","the","function","input","arguments","into","the","hyper-parameter","section","as","kwargs",",","and","storing","function","results","as","named","artifacts","Example",":","def","mock_func","(","a=6",",","b=9",")",":","c","=","a","*","b","print","(","a",",","b",",","c",")","return","c",",","c","*","*","2","create_task_from_function","(","mock_func",",","function_return=","[","'mul","'",",","'square","'","]",")","Example","arguments","from","other","Tasks","(","artifact",")",":","def","mock_func","(","matrix_np",")",":","c","=","matrix_np","*","matrix_np","print","(","matrix_np",",","c",")","return","c","create_task_from_function","(","mock_func",",","function_input_artifacts=","{","'matrix_np","'",":","'aabb1122.previous_matrix","'","}",",","function_return=","[","'square_matrix","'","]",")",":","param","a_function",":","A","global","function","to","convert","into","a","standalone","Task",":","param","function_kwargs",":","Optional",",","provide","subset","of","function","arguments","and","default","values","to","expose",".","If","not","provided","automatically","take","all","function","arguments","&","defaults",":","param","function_input_artifacts",":","Optional",",","pass","input","arguments","to","the","function","from","other","Tasks","'s","output","artifact",".","Example","argument","named","`","numpy_matrix","`","from","Task","ID","`","aabbcc","`","artifact","name","`","answer","`",":","{","'numpy_matrix","'",":","'aabbcc.answer","'","}",":","param","function_return",":","Provide","a","list","of","names","for","all","the","results",".","If","not","provided","no","results","will","be","stored","as","artifacts",".",":","param","project_name",":","Set","the","project","name","for","the","task",".","Required","if","base_task_id","is","None",".",":","param","task_name",":","Set","the","name","of","the","remote","task",".","Required","if","base_task_id","is","None",".",":","param","task_type",":","Optional",",","The","task","type","to","be","created",".","Supported","values",":","'training","'",",","'testing","'",",","'inference","'",",","'data_processing","'",",","'application","'",",","'monitor","'",",","'controller","'",",","'optimizer","'",",","'service","'",",","'qc","'",",","'custom'",":","param","auto_connect_frameworks",":","Control","the","frameworks","auto","connect",",","see","`","Task.init","`","auto_connect_frameworks",":","param","auto_connect_arg_parser",":","Control","the","ArgParser","auto","connect",",","see","`","Task.init","`","auto_connect_arg_parser",":","param","repo",":","Remote","URL","for","the","repository","to","use",",","OR","path","to","local","copy","of","the","git","repository","Example",":","'https",":","\/\/github.com\/allegroai\/clearml.git","'","or","'~\/project\/repo'",":","param","branch",":","Select","specific","repository","branch\/tag","(","implies","the","latest","commit","from","the","branch",")",":","param","commit",":","Select","specific","commit","id","to","use","(","default",":","latest","commit",",","or","when","used","with","local","repository","matching","the","local","commit","id",")",":","param","packages",":","Manually","specify","a","list","of","required","packages","or","a","local","requirements.txt","file",".","Example",":","[","``","tqdm",">","=2.1","''",",","``","scikit-learn","''","]","or","``",".\/requirements.txt","''","If","not","provided",",","packages","are","automatically","added","based","on","the","imports","used","in","the","function",".",":","param","docker",":","Select","the","docker","image","to","be","executed","in","by","the","remote","session",":","param","docker_args",":","Add","docker","arguments",",","pass","a","single","string",":","param","docker_bash_setup_script",":","Add","bash","script","to","be","executed","inside","the","docker","before","setting","up","the","Task","'s","environment",":","param","output_uri",":","Optional",",","set","the","Tasks","'s","output_uri","(","Storage","destination",")",".","examples",":","'s3",":","\/\/bucket\/folder","'",",","'https",":","\/\/server\/","'",",","'gs",":","\/\/bucket\/folder","'",",","'azure",":","\/\/bucket","'",",","'\/folder\/'",":","param","helper_functions",":","Optional",",","a","list","of","helper","functions","to","make","available","for","the","standalone","function","Task",".",":","param","dry_run",":","If","True",",","do","not","create","the","Task",",","but","return","a","dict","of","the","Task","'s","definitions",":","param","task_template_header",":","A","string","placed","at","the","top","of","the","task","'s","code",":","param","artifact_serialization_function",":","A","serialization","function","that","takes","one","parameter","of","any","type","which","is","the","object","to","be","serialized",".","The","function","should","return","a","`","bytes","`","or","`","bytearray","`","object",",","which","represents","the","serialized","object",".","All","parameter\/return","artifacts","uploaded","by","the","pipeline","will","be","serialized","using","this","function",".","All","relevant","imports","must","be","done","in","this","function",".","For","example",":","..","code-block",":",":","py","def","serialize","(","obj",")",":","import","dill","return","dill.dumps","(","obj",")",":","param","artifact_deserialization_function",":","A","deserialization","function","that","takes","one","parameter","of","type","`","bytes","`",",","which","represents","the","serialized","object",".","This","function","should","return","the","deserialized","object",".","All","parameter\/return","artifacts","fetched","by","the","pipeline","will","be","deserialized","using","this","function",".","All","relevant","imports","must","be","done","in","this","function",".","For","example",":","..","code-block",":",":","py","def","deserialize","(","bytes_",")",":","import","dill","return","dill.loads","(","bytes_",")",":","param","_sanitize_function",":","Sanitization","function","for","the","function","string",".",":","param","_sanitize_helper_functions",":","Sanitization","function","for","the","helper","function","string",".",":","param","skip_global_imports",":","If","True",",","the","global","imports","will","not","be","fetched","from","the","function","'s","file",",","otherwise","all","global","imports","will","be","automatically","imported","in","a","safe","manner","at","the","beginning","of","the","function's","execution",".","Default","is","False",":","return",":","Newly","created","Task","object"]
code: def create_task_from_function(
      cls,
      a_function,  # type: Callable
      function_kwargs=None,  # type: Optional[Dict[str, Any]]
      function_input_artifacts=None,  # type: Optional[Dict[str, str]]
      function_return=None,  # type: Optional[List[str]]
      project_name=None,  # type: Optional[str]
      task_name=None,  # type: Optional[str]
      task_type=None,  # type: Optional[str]
      auto_connect_frameworks=None,  # type: Optional[dict]
      auto_connect_arg_parser=None,  # type: Optional[dict]
      repo=None,  # type: Optional[str]
      branch=None,  # type: Optional[str]
      commit=None,  # type: Optional[str]
      packages=None,  # type: Optional[Union[str, Sequence[str]]]
      docker=None,  # type: Optional[str]
      docker_args=None,  # type: Optional[str]
      docker_bash_setup_script=None,  # type: Optional[str]
      output_uri=None,  # type: Optional[str]
      helper_functions=None,  # type: Optional[Sequence[Callable]]
      dry_run=False,  # type: bool
      task_template_header=None,  # type: Optional[str]
      artifact_serialization_function=None,  # type: Optional[Callable[[Any], Union[bytes, bytearray]]]
      artifact_deserialization_function=None,  # type: Optional[Callable[[bytes], Any]]
      _sanitize_function=None,  # type: Optional[Callable[[str], str]]
      _sanitize_helper_functions=None,  # type: Optional[Callable[[str], str]]
      skip_global_imports=False,  # type: bool
  ):
      # type: (...) -> Optional[Dict, Task]
      """
      Create a Task from a function, including wrapping the function input arguments
      into the hyper-parameter section as kwargs, and storing function results as named artifacts
  
      Example:
          def mock_func(a=6, b=9):
              c = a*b
              print(a, b, c)
              return c, c**2
  
          create_task_from_function(mock_func, function_return=['mul', 'square'])
  
      Example arguments from other Tasks (artifact):
          def mock_func(matrix_np):
              c = matrix_np*matrix_np
              print(matrix_np, c)
              return c
  
          create_task_from_function(
              mock_func,
              function_input_artifacts={'matrix_np': 'aabb1122.previous_matrix'},
              function_return=['square_matrix']
          )
  
      :param a_function: A global function to convert into a standalone Task
      :param function_kwargs: Optional, provide subset of function arguments and default values to expose.
          If not provided automatically take all function arguments & defaults
      :param function_input_artifacts: Optional, pass input arguments to the function from
          other Tasks's output artifact.
          Example argument named `numpy_matrix` from Task ID `aabbcc` artifact name `answer`:
          {'numpy_matrix': 'aabbcc.answer'}
      :param function_return: Provide a list of names for all the results.
          If not provided no results will be stored as artifacts.
      :param project_name: Set the project name for the task. Required if base_task_id is None.
      :param task_name: Set the name of the remote task. Required if base_task_id is None.
      :param task_type: Optional, The task type to be created. Supported values: 'training', 'testing', 'inference',
          'data_processing', 'application', 'monitor', 'controller', 'optimizer', 'service', 'qc', 'custom'
      :param auto_connect_frameworks: Control the frameworks auto connect, see `Task.init` auto_connect_frameworks
      :param auto_connect_arg_parser: Control the ArgParser auto connect, see `Task.init` auto_connect_arg_parser
      :param repo: Remote URL for the repository to use, OR path to local copy of the git repository
          Example: 'https://github.com/allegroai/clearml.git' or '~/project/repo'
      :param branch: Select specific repository branch/tag (implies the latest commit from the branch)
      :param commit: Select specific commit id to use (default: latest commit,
          or when used with local repository matching the local commit id)
      :param packages: Manually specify a list of required packages or a local requirements.txt file.
          Example: ["tqdm>=2.1", "scikit-learn"] or "./requirements.txt"
          If not provided, packages are automatically added based on the imports used in the function.
      :param docker: Select the docker image to be executed in by the remote session
      :param docker_args: Add docker arguments, pass a single string
      :param docker_bash_setup_script: Add bash script to be executed
          inside the docker before setting up the Task's environment
      :param output_uri: Optional, set the Tasks's output_uri (Storage destination).
          examples: 's3://bucket/folder', 'https://server/' , 'gs://bucket/folder', 'azure://bucket', '/folder/'
      :param helper_functions: Optional, a list of helper functions to make available
          for the standalone function Task.
      :param dry_run: If True, do not create the Task, but return a dict of the Task's definitions
      :param task_template_header: A string placed at the top of the task's code
      :param artifact_serialization_function: A serialization function that takes one
          parameter of any type which is the object to be serialized. The function should return
          a `bytes` or `bytearray` object, which represents the serialized object. All parameter/return
          artifacts uploaded by the pipeline will be serialized using this function.
          All relevant imports must be done in this function. For example:
  
          .. code-block:: py
  
              def serialize(obj):
                  import dill
                  return dill.dumps(obj)
      :param artifact_deserialization_function: A deserialization function that takes one parameter of type `bytes`,
          which represents the serialized object. This function should return the deserialized object.
          All parameter/return artifacts fetched by the pipeline will be deserialized using this function.
          All relevant imports must be done in this function. For example:
  
          .. code-block:: py
  
              def deserialize(bytes_):
                  import dill
                  return dill.loads(bytes_)
      :param _sanitize_function: Sanitization function for the function string.
      :param _sanitize_helper_functions: Sanitization function for the helper function string.
      :param skip_global_imports: If True, the global imports will not be fetched from the function's file, otherwise
          all global imports will be automatically imported in a safe manner at the beginning of the function's
          execution. Default is False
      :return: Newly created Task object
      """
      # not set -> equals True
      if auto_connect_frameworks is None:
          auto_connect_frameworks = True
      if auto_connect_arg_parser is None:
          auto_connect_arg_parser = True
  
      assert not auto_connect_frameworks or isinstance(
          auto_connect_frameworks, (bool, dict)
      )
      assert not auto_connect_arg_parser or isinstance(
          auto_connect_arg_parser, (bool, dict)
      )
  
      (
          function_source,
          function_name,
      ) = CreateFromFunction.__extract_function_information(
          a_function,
          sanitize_function=_sanitize_function,
          skip_global_imports=skip_global_imports,
      )
      # add helper functions on top.
      for f in helper_functions or []:
          (
              helper_function_source,
              _,
          ) = CreateFromFunction.__extract_function_information(
              f, sanitize_function=_sanitize_helper_functions
          )
          function_source = (
              helper_function_source + "\n\n" + function_source
          )
  
      (
          artifact_serialization_function_source,
          artifact_serialization_function_name,
      ) = (
          CreateFromFunction.__extract_function_information(
              artifact_serialization_function
          )
          if artifact_serialization_function
          else ("", "None")
      )
      (
          artifact_deserialization_function_source,
          artifact_deserialization_function_name,
      ) = (
          CreateFromFunction.__extract_function_information(
              artifact_deserialization_function
          )
          if artifact_deserialization_function
          else ("", "None")
      )
  
      function_input_artifacts = function_input_artifacts or dict()
      # verify artifact kwargs:
      if not all(
          len(v.split(".", 1)) == 2
          for v in function_input_artifacts.values()
      ):
          raise ValueError(
              "function_input_artifacts={}, it must in the format: "
              '{{"argument": "task_id.artifact_name"}}'.format(
                  function_input_artifacts
              )
          )
      inspect_args = None
      function_kwargs_types = dict()
      if function_kwargs is None:
          function_kwargs = dict()
          inspect_args = inspect.getfullargspec(a_function)
          if inspect_args and inspect_args.args:
              inspect_defaults_vals = inspect_args.defaults
              inspect_defaults_args = inspect_args.args
  
              # adjust the defaults so they match the args (match from the end)
              if inspect_defaults_vals and len(
                  inspect_defaults_vals
              ) != len(inspect_defaults_args):
                  inspect_defaults_args = inspect_defaults_args[
                      -len(inspect_defaults_vals) :
                  ]
  
              if inspect_defaults_vals and len(
                  inspect_defaults_vals
              ) != len(inspect_defaults_args):
                  getLogger().warning(
                      "Ignoring default argument values: "
                      "could not find all default valued for: '{}'".format(
                          function_name
                      )
                  )
                  inspect_defaults_vals = []
  
              function_kwargs = (
                  {
                      str(k): v
                      for k, v in zip(
                          inspect_defaults_args, inspect_defaults_vals
                      )
                  }
                  if inspect_defaults_vals
                  else {str(k): None for k in inspect_defaults_args}
              )
  
      if function_kwargs:
          if not inspect_args:
              inspect_args = inspect.getfullargspec(a_function)
          # inspect_func.annotations[k]
          if inspect_args.annotations:
              supported_types = _Arguments.get_supported_types()
              function_kwargs_types = {
                  str(k): str(inspect_args.annotations[k].__name__)
                  for k in inspect_args.annotations
                  if inspect_args.annotations[k] in supported_types
              }
  
      task_template = cls.task_template.format(
          header=task_template_header
          or cls.default_task_template_header,
          auto_connect_frameworks=auto_connect_frameworks,
          auto_connect_arg_parser=auto_connect_arg_parser,
          kwargs_section=cls.kwargs_section,
          input_artifact_section=cls.input_artifact_section,
          function_source=function_source,
          function_kwargs=function_kwargs,
          function_input_artifacts=function_input_artifacts,
          function_name=function_name,
          function_return=function_return,
          return_section=cls.return_section,
          artifact_serialization_function_source=artifact_serialization_function_source,
          artifact_serialization_function_name=artifact_serialization_function_name,
          artifact_deserialization_function_source=artifact_deserialization_function_source,
          artifact_deserialization_function_name=artifact_deserialization_function_name,
      )
  
      temp_dir = repo if repo and os.path.isdir(repo) else None
      with tempfile.NamedTemporaryFile(
          "w", suffix=".py", dir=temp_dir
      ) as temp_file:
          temp_file.write(task_template)
          temp_file.flush()
  
          requirements_file = None
          if (
              packages
              and not isinstance(packages, (list, tuple))
              and Path(packages).is_file()
          ):
              requirements_file = packages
              packages = False
  
          populate = CreateAndPopulate(
              project_name=project_name,
              task_name=task_name or str(function_name),
              task_type=task_type,
              script=temp_file.name,
              packages=packages if packages is not None else True,
              requirements_file=requirements_file,
              repo=repo,
              branch=branch,
              commit=commit,
              docker=docker,
              docker_args=docker_args,
              docker_bash_setup_script=docker_bash_setup_script,
              output_uri=output_uri,
              add_task_init_call=False,
          )
          entry_point = "{}.py".format(function_name)
          task = populate.create_task(dry_run=dry_run)
  
          if dry_run:
              task["script"]["diff"] = task_template
              task["script"]["entry_point"] = entry_point
              task["script"]["working_dir"] = "."
              task["hyperparams"] = {
                  cls.kwargs_section: {
                      k: dict(
                          section=cls.kwargs_section,
                          name=k,
                          value=str(v) if v is not None else "",
                          type=function_kwargs_types.get(k, None),
                      )
                      for k, v in (function_kwargs or {}).items()
                  },
                  cls.input_artifact_section: {
                      k: dict(
                          section=cls.input_artifact_section,
                          name=k,
                          value=str(v) if v is not None else "",
                      )
                      for k, v in (
                          function_input_artifacts or {}
                      ).items()
                  },
              }
          else:
              task.update_task(
                  task_data={
                      "script": task.data.script.to_dict().update(
                          {
                              "entry_point": entry_point,
                              "working_dir": ".",
                              "diff": task_template,
                          }
                      )
                  }
              )
              hyper_parameters = (
                  {
                      "{}/{}".format(cls.kwargs_section, k): str(v)
                      for k, v in function_kwargs
                  }
                  if function_kwargs
                  else {}
              )
              hyper_parameters.update(
                  {
                      "{}/{}".format(
                          cls.input_artifact_section, k
                      ): str(v)
                      for k, v in function_input_artifacts
                  }
                  if function_input_artifacts
                  else {}
              )
              __function_kwargs_types = (
                  {
                      "{}/{}".format(cls.kwargs_section, k): v
                      for k, v in function_kwargs_types
                  }
                  if function_kwargs_types
                  else None
              )
              task.set_parameters(
                  hyper_parameters,
                  __parameters_types=__function_kwargs_types,
              )
  
          return task
  
imports: import inspect
  import json
  import os
  import re
  import tempfile
  from sys import platform
  from functools import reduce
  from logging import getLogger
  from typing import Optional, Sequence, Union, Tuple, List, Callable, Dict, Any
  from pathlib2 import Path
  from six.moves.urllib.parse import urlparse
  from .args import _Arguments
  from .repo import ScriptInfo
  from ...task import Task
label_desc: Use image node_id 1 for calling the CreateFromFunction obj's underlying member method code with example usage: obj.create_task_from_function(cls, a_function, function_kwargs, function_input_artifacts, function_return, project_name, task_name, task_type, auto_connect_frameworks, auto_connect_arg_parser, repo, branch, commit, packages, docker, docker_args, docker_bash_setup_script, output_uri, helper_functions, dry_run, task_template_header, artifact_serialization_function, artifact_deserialization_function, _sanitize_function, _sanitize_helper_functions, skip_global_imports) and returns: task