We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Running into TypeError exception with `batch_predict` component
### What steps did you take:
I am using batch_predict component to schedule prediction job on AI platform. I was referring to steps here. https://github.com/kubeflow/pipelines/tree/master/components/gcp/ml_engine/batch_predict
- I downloaded the component from https://raw.githubusercontent.com/kubeflow/pipelines/1.0.0/components/gcp/ml_engine/batch_predict/component.yaml. 
- loaded the component with `ml_op = load_component_from_file(<local-path>)`
- created pipeline component with `ml_op(<attributes-mentioned-below>)`
- scheduled the pipeline on kubeflow

Job Parameters:

```
prediction_input: {'runtimeVersion': '2.1'}
project_id: <gcs-project>
region: <region-name>
model_path: <gcs-path>
input_paths: [<list-of-input-paths>]
input_data_format: TF_RECORD_GZIP
output_path: <gcs-path>
output_data_format: JSON
```

### What happened:

Running into type error while generating the job_id.

Issue:

It is raising an exception while generating id to schedule the job. This line is failing https://github.com/kubeflow/pipelines/blob/1.0.0/components/gcp/container/component_sdk/python/kfp_component/google/common/_utils.py#L44 with this error `TypeError: expected string or buffer`. I am not passing any job_id and job_id_prefix. It is using the default values for generating the name.

Error trace:

```
INFO:root:Start KFP context with ID: 858b3ff01cdeed5c0b0b7fd9d2655641.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/ml/kfp_component/launcher/__main__.py", line 45, in <module>
    main()
  File "/ml/kfp_component/launcher/__main__.py", line 42, in main
    launch(args.file_or_module, args.args)
  File "kfp_component/launcher/launcher.py", line 45, in launch
    return fire.Fire(module, command=args, name=module.__name__)
  File "/usr/local/lib/python2.7/site-packages/fire/core.py", line 127, in Fire
    component_trace = _Fire(component, args, context, name)
  File "/usr/local/lib/python2.7/site-packages/fire/core.py", line 366, in _Fire
    component, remaining_args)
  File "/usr/local/lib/python2.7/site-packages/fire/core.py", line 542, in _CallCallable
    result = fn(*varargs, **kwargs)
  File "kfp_component/google/ml_engine/_batch_predict.py", line 70, in batch_predict
    create_job(project_id, job, job_id_prefix, wait_interval)
  File "kfp_component/google/ml_engine/_create_job.py", line 48, in create_job
    return CreateJobOp(project_id, job, job_id_prefix, job_id, wait_interval
  File "kfp_component/google/ml_engine/_create_job.py", line 63, in execute_and_wait
    self._set_job_id(ctx.context_id())
  File "kfp_component/google/ml_engine/_create_job.py", line 74, in _set_job_id
    job_id = gcp_common.normalize_name(job_id)
  File "kfp_component/google/common/_utils.py", line 45, in normalize_name
    invalid_char_placeholder, name)
  File "/usr/local/lib/python2.7/re.py", line 155, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer

```

### What did you expect to happen:

I was expecting the component to schedule the job with auto generated job id.

### Environment:
<!-- Please fill in those that seem relevant. -->

How did you deploy Kubeflow Pipelines (KFP)?
- I used Kustomize to deploy standalone kubeflow pipeline manifests on GKE.

<!-- If you are not sure, here's [an introduction of all options](https://www.kubeflow.org/docs/pipelines/installation/overview/). -->

KFP version: <!-- If you are not sure, build commit shows on bottom of KFP UI left sidenav. -->
1.0.0

KFP SDK version: <!-- Please attach the output of this shell command: $pip list | grep kfp -->
kfp==0.5.1

### Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

/kind bug
<!-- Please include labels by uncommenting them to help us better triage issues, choose from the following -->
<!--
// /area frontend
 /area backend
// /area sdk
// /area testing
 /area engprod
-->


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py`
Content:
```
1 # Copyright 2018 Google LLC
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #      http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 from fire import decorators
16 from ._create_job import create_job
17 
18 @decorators.SetParseFns(python_version=str, runtime_version=str)
19 def train(project_id, job_id_output_path, python_module=None, package_uris=None, 
20     region=None, args=None, job_dir=None, python_version=None, 
21     runtime_version=None, master_image_uri=None, worker_image_uri=None, 
22     training_input=None, job_id_prefix=None, job_id=None, wait_interval=30):
23     """Creates a MLEngine training job.
24 
25     Args:
26         project_id (str): Required. The ID of the parent project of the job.
27         python_module (str): Required. The Python module name to run after 
28             installing the packages.
29         package_uris (list): Required. The Google Cloud Storage location of 
30             the packages with the training program and any additional 
31             dependencies. The maximum number of package URIs is 100.
32         region (str): Required. The Google Compute Engine region to run the 
33             training job in
34         args (list): Command line arguments to pass to the program.
35         job_dir (str): A Google Cloud Storage path in which to store training 
36             outputs and other data needed for training. This path is passed 
37             to your TensorFlow program as the '--job-dir' command-line 
38             argument. The benefit of specifying this field is that Cloud ML 
39             validates the path for use in training.
40         python_version (str): Optional. The version of Python used in training. 
41             If not set, the default version is '2.7'. Python '3.5' is 
42             available when runtimeVersion is set to '1.4' and above. 
43             Python '2.7' works with all supported runtime versions.
44         runtime_version (str): Optional. The Cloud ML Engine runtime version 
45             to use for training. If not set, Cloud ML Engine uses the 
46             default stable version, 1.0. 
47         master_image_uri (str): The Docker image to run on the master replica. 
48             This image must be in Container Registry.
49         worker_image_uri (str): The Docker image to run on the worker replica. 
50             This image must be in Container Registry.
51         training_input (dict): Input parameters to create a training job.
52         job_id_prefix (str): the prefix of the generated job id.
53         job_id (str): the created job_id, takes precedence over generated job
54             id if set.
55         wait_interval (int): optional wait interval between calls
56             to get job status. Defaults to 30.
57     """
58     if not training_input:
59         training_input = {}
60     if python_module:
61         training_input['pythonModule'] = python_module
62     if package_uris:
63         training_input['packageUris'] = package_uris
64     if region:
65         training_input['region'] = region
66     if args:
67         training_input['args'] = args
68     if job_dir:
69         training_input['jobDir'] = job_dir
70     if python_version:
71         training_input['pythonVersion'] = python_version
72     if runtime_version:
73         training_input['runtimeVersion'] = runtime_version
74     if master_image_uri:
75         if 'masterConfig' not in training_input:
76             training_input['masterConfig'] = {}
77         training_input['masterConfig']['imageUri'] = master_image_uri
78     if worker_image_uri:
79         if 'workerConfig' not in training_input:
80             training_input['workerConfig'] = {}
81         training_input['workerConfig']['imageUri'] = worker_image_uri
82     job = {
83         'trainingInput': training_input
84     }
85     return create_job(project_id, job, job_id_prefix, job_id, wait_interval, job_id_output_path=job_id_output_path)
```

Path: `components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py`
Content:
```
1 # Copyright 2018 Google LLC
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #      http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import re
16 
17 from ._create_job import create_job
18 
19 def batch_predict(project_id, model_path, input_paths, input_data_format, 
20     output_path, region, job_id_output_path, output_data_format=None, prediction_input=None, job_id_prefix=None,
21     wait_interval=30):
22     """Creates a MLEngine batch prediction job.
23 
24     Args:
25         project_id (str): Required. The ID of the parent project of the job.
26         model_path (str): Required. The path to the model. It can be either:
27             `projects/[PROJECT_ID]/models/[MODEL_ID]` or 
28             `projects/[PROJECT_ID]/models/[MODEL_ID]/versions/[VERSION_ID]`
29             or a GCS path of a model file.
30         input_paths (list): Required. The Google Cloud Storage location of 
31             the input data files. May contain wildcards.
32         input_data_format (str): Required. The format of the input data files.
33             See https://cloud.google.com/ml-engine/reference/rest/v1/projects.jobs#DataFormat.
34         output_path (str): Required. The output Google Cloud Storage location.
35         region (str): Required. The Google Compute Engine region to run the 
36             prediction job in.
37         output_data_format (str): Optional. Format of the output data files, 
38             defaults to JSON.
39         prediction_input (dict): Input parameters to create a prediction job.
40         job_id_prefix (str): the prefix of the generated job id.
41         wait_interval (int): optional wait interval between calls
42             to get job status. Defaults to 30.
43     """
44     if not prediction_input:
45         prediction_input = {}
46     if not model_path:
47         raise ValueError('model_path must be provided.')
48     if _is_model_name(model_path):
49         prediction_input['modelName'] = model_path
50     elif _is_model_version_name(model_path):
51         prediction_input['versionName'] = model_path
52     elif _is_gcs_path(model_path):
53         prediction_input['uri'] = model_path
54     else:
55         raise ValueError('model_path value is invalid.')
56     
57     if input_paths:
58         prediction_input['inputPaths'] = input_paths
59     if input_data_format:
60         prediction_input['dataFormat'] = input_data_format
61     if output_path:
62         prediction_input['outputPath'] = output_path
63     if output_data_format:
64         prediction_input['outputDataFormat'] = output_data_format
65     if region:
66         prediction_input['region'] = region
67     job = {
68         'predictionInput': prediction_input
69     }
70     create_job(project_id, job, job_id_prefix, wait_interval, job_id_output_path=job_id_output_path)
71     
72 def _is_model_name(name):
73     return re.match(r'/projects/[^/]+/models/[^/]+$', name)
74 
75 def _is_model_version_name(name):
76     return re.match(r'/projects/[^/]+/models/[^/]+/versions/[^/]+$', name)
77 
78 def _is_gcs_path(name):
79     return name.startswith('gs://')
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py
--- a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py
+++ b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py
@@ -67,7 +67,13 @@
     job = {
         'predictionInput': prediction_input
     }
-    create_job(project_id, job, job_id_prefix, wait_interval, job_id_output_path=job_id_output_path)
+    create_job(
+        project_id=project_id,
+        job=job,
+        job_id_prefix=job_id_prefix,
+        wait_interval=wait_interval,
+        job_id_output_path=job_id_output_path,
+    )
     
 def _is_model_name(name):
     return re.match(r'/projects/[^/]+/models/[^/]+$', name)
diff --git a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py
--- a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py
+++ b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py
@@ -82,4 +82,11 @@
     job = {
         'trainingInput': training_input
     }
-    return create_job(project_id, job, job_id_prefix, job_id, wait_interval, job_id_output_path=job_id_output_path)
\ No newline at end of file
+    return create_job(
+        project_id=project_id,
+        job=job,
+        job_id_prefix=job_id_prefix,
+        job_id=job_id,
+        wait_interval=wait_interval,
+        job_id_output_path=job_id_output_path,
+    )
\ No newline at end of file

{"golden_diff": "diff --git a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py\n--- a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py\n+++ b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py\n@@ -67,7 +67,13 @@\n     job = {\n         'predictionInput': prediction_input\n     }\n-    create_job(project_id, job, job_id_prefix, wait_interval, job_id_output_path=job_id_output_path)\n+    create_job(\n+        project_id=project_id,\n+        job=job,\n+        job_id_prefix=job_id_prefix,\n+        wait_interval=wait_interval,\n+        job_id_output_path=job_id_output_path,\n+    )\n     \n def _is_model_name(name):\n     return re.match(r'/projects/[^/]+/models/[^/]+$', name)\ndiff --git a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py\n--- a/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py\n+++ b/components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py\n@@ -82,4 +82,11 @@\n     job = {\n         'trainingInput': training_input\n     }\n-    return create_job(project_id, job, job_id_prefix, job_id, wait_interval, job_id_output_path=job_id_output_path)\n\\ No newline at end of file\n+    return create_job(\n+        project_id=project_id,\n+        job=job,\n+        job_id_prefix=job_id_prefix,\n+        job_id=job_id,\n+        wait_interval=wait_interval,\n+        job_id_output_path=job_id_output_path,\n+    )\n\\ No newline at end of file\n", "issue": "Running into TypeError exception with `batch_predict` component\n### What steps did you take:\r\nI am using batch_predict component to schedule prediction job on AI platform. I was referring to steps here. https://github.com/kubeflow/pipelines/tree/master/components/gcp/ml_engine/batch_predict\r\n- I downloaded the component from https://raw.githubusercontent.com/kubeflow/pipelines/1.0.0/components/gcp/ml_engine/batch_predict/component.yaml. \r\n- loaded the component with `ml_op = load_component_from_file(<local-path>)`\r\n- created pipeline component with `ml_op(<attributes-mentioned-below>)`\r\n- scheduled the pipeline on kubeflow\r\n\r\nJob Parameters:\r\n\r\n```\r\nprediction_input: {'runtimeVersion': '2.1'}\r\nproject_id: <gcs-project>\r\nregion: <region-name>\r\nmodel_path: <gcs-path>\r\ninput_paths: [<list-of-input-paths>]\r\ninput_data_format: TF_RECORD_GZIP\r\noutput_path: <gcs-path>\r\noutput_data_format: JSON\r\n```\r\n\r\n### What happened:\r\n\r\nRunning into type error while generating the job_id.\r\n\r\nIssue:\r\n\r\nIt is raising an exception while generating id to schedule the job. This line is failing https://github.com/kubeflow/pipelines/blob/1.0.0/components/gcp/container/component_sdk/python/kfp_component/google/common/_utils.py#L44 with this error `TypeError: expected string or buffer`. I am not passing any job_id and job_id_prefix. It is using the default values for generating the name.\r\n\r\nError trace:\r\n\r\n```\r\nINFO:root:Start KFP context with ID: 858b3ff01cdeed5c0b0b7fd9d2655641.\r\nTraceback (most recent call last):\r\n  File \"/usr/local/lib/python2.7/runpy.py\", line 174, in _run_module_as_main\r\n    \"__main__\", fname, loader, pkg_name)\r\n  File \"/usr/local/lib/python2.7/runpy.py\", line 72, in _run_code\r\n    exec code in run_globals\r\n  File \"/ml/kfp_component/launcher/__main__.py\", line 45, in <module>\r\n    main()\r\n  File \"/ml/kfp_component/launcher/__main__.py\", line 42, in main\r\n    launch(args.file_or_module, args.args)\r\n  File \"kfp_component/launcher/launcher.py\", line 45, in launch\r\n    return fire.Fire(module, command=args, name=module.__name__)\r\n  File \"/usr/local/lib/python2.7/site-packages/fire/core.py\", line 127, in Fire\r\n    component_trace = _Fire(component, args, context, name)\r\n  File \"/usr/local/lib/python2.7/site-packages/fire/core.py\", line 366, in _Fire\r\n    component, remaining_args)\r\n  File \"/usr/local/lib/python2.7/site-packages/fire/core.py\", line 542, in _CallCallable\r\n    result = fn(*varargs, **kwargs)\r\n  File \"kfp_component/google/ml_engine/_batch_predict.py\", line 70, in batch_predict\r\n    create_job(project_id, job, job_id_prefix, wait_interval)\r\n  File \"kfp_component/google/ml_engine/_create_job.py\", line 48, in create_job\r\n    return CreateJobOp(project_id, job, job_id_prefix, job_id, wait_interval\r\n  File \"kfp_component/google/ml_engine/_create_job.py\", line 63, in execute_and_wait\r\n    self._set_job_id(ctx.context_id())\r\n  File \"kfp_component/google/ml_engine/_create_job.py\", line 74, in _set_job_id\r\n    job_id = gcp_common.normalize_name(job_id)\r\n  File \"kfp_component/google/common/_utils.py\", line 45, in normalize_name\r\n    invalid_char_placeholder, name)\r\n  File \"/usr/local/lib/python2.7/re.py\", line 155, in sub\r\n    return _compile(pattern, flags).sub(repl, string, count)\r\nTypeError: expected string or buffer\r\n\r\n```\r\n\r\n### What did you expect to happen:\r\n\r\nI was expecting the component to schedule the job with auto generated job id.\r\n\r\n### Environment:\r\n<!-- Please fill in those that seem relevant. -->\r\n\r\nHow did you deploy Kubeflow Pipelines (KFP)?\r\n- I used Kustomize to deploy standalone kubeflow pipeline manifests on GKE.\r\n\r\n<!-- If you are not sure, here's [an introduction of all options](https://www.kubeflow.org/docs/pipelines/installation/overview/). -->\r\n\r\nKFP version: <!-- If you are not sure, build commit shows on bottom of KFP UI left sidenav. -->\r\n1.0.0\r\n\r\nKFP SDK version: <!-- Please attach the output of this shell command: $pip list | grep kfp -->\r\nkfp==0.5.1\r\n\r\n### Anything else you would like to add:\r\n[Miscellaneous information that will assist in solving the issue.]\r\n\r\n/kind bug\r\n<!-- Please include labels by uncommenting them to help us better triage issues, choose from the following -->\r\n<!--\r\n// /area frontend\r\n /area backend\r\n// /area sdk\r\n// /area testing\r\n /area engprod\r\n-->\r\n\n", "before_files": [{"content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom fire import decorators\nfrom ._create_job import create_job\n\n@decorators.SetParseFns(python_version=str, runtime_version=str)\ndef train(project_id, job_id_output_path, python_module=None, package_uris=None, \n    region=None, args=None, job_dir=None, python_version=None, \n    runtime_version=None, master_image_uri=None, worker_image_uri=None, \n    training_input=None, job_id_prefix=None, job_id=None, wait_interval=30):\n    \"\"\"Creates a MLEngine training job.\n\n    Args:\n        project_id (str): Required. The ID of the parent project of the job.\n        python_module (str): Required. The Python module name to run after \n            installing the packages.\n        package_uris (list): Required. The Google Cloud Storage location of \n            the packages with the training program and any additional \n            dependencies. The maximum number of package URIs is 100.\n        region (str): Required. The Google Compute Engine region to run the \n            training job in\n        args (list): Command line arguments to pass to the program.\n        job_dir (str): A Google Cloud Storage path in which to store training \n            outputs and other data needed for training. This path is passed \n            to your TensorFlow program as the '--job-dir' command-line \n            argument. The benefit of specifying this field is that Cloud ML \n            validates the path for use in training.\n        python_version (str): Optional. The version of Python used in training. \n            If not set, the default version is '2.7'. Python '3.5' is \n            available when runtimeVersion is set to '1.4' and above. \n            Python '2.7' works with all supported runtime versions.\n        runtime_version (str): Optional. The Cloud ML Engine runtime version \n            to use for training. If not set, Cloud ML Engine uses the \n            default stable version, 1.0. \n        master_image_uri (str): The Docker image to run on the master replica. \n            This image must be in Container Registry.\n        worker_image_uri (str): The Docker image to run on the worker replica. \n            This image must be in Container Registry.\n        training_input (dict): Input parameters to create a training job.\n        job_id_prefix (str): the prefix of the generated job id.\n        job_id (str): the created job_id, takes precedence over generated job\n            id if set.\n        wait_interval (int): optional wait interval between calls\n            to get job status. Defaults to 30.\n    \"\"\"\n    if not training_input:\n        training_input = {}\n    if python_module:\n        training_input['pythonModule'] = python_module\n    if package_uris:\n        training_input['packageUris'] = package_uris\n    if region:\n        training_input['region'] = region\n    if args:\n        training_input['args'] = args\n    if job_dir:\n        training_input['jobDir'] = job_dir\n    if python_version:\n        training_input['pythonVersion'] = python_version\n    if runtime_version:\n        training_input['runtimeVersion'] = runtime_version\n    if master_image_uri:\n        if 'masterConfig' not in training_input:\n            training_input['masterConfig'] = {}\n        training_input['masterConfig']['imageUri'] = master_image_uri\n    if worker_image_uri:\n        if 'workerConfig' not in training_input:\n            training_input['workerConfig'] = {}\n        training_input['workerConfig']['imageUri'] = worker_image_uri\n    job = {\n        'trainingInput': training_input\n    }\n    return create_job(project_id, job, job_id_prefix, job_id, wait_interval, job_id_output_path=job_id_output_path)", "path": "components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py"}, {"content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport re\n\nfrom ._create_job import create_job\n\ndef batch_predict(project_id, model_path, input_paths, input_data_format, \n    output_path, region, job_id_output_path, output_data_format=None, prediction_input=None, job_id_prefix=None,\n    wait_interval=30):\n    \"\"\"Creates a MLEngine batch prediction job.\n\n    Args:\n        project_id (str): Required. The ID of the parent project of the job.\n        model_path (str): Required. The path to the model. It can be either:\n            `projects/[PROJECT_ID]/models/[MODEL_ID]` or \n            `projects/[PROJECT_ID]/models/[MODEL_ID]/versions/[VERSION_ID]`\n            or a GCS path of a model file.\n        input_paths (list): Required. The Google Cloud Storage location of \n            the input data files. May contain wildcards.\n        input_data_format (str): Required. The format of the input data files.\n            See https://cloud.google.com/ml-engine/reference/rest/v1/projects.jobs#DataFormat.\n        output_path (str): Required. The output Google Cloud Storage location.\n        region (str): Required. The Google Compute Engine region to run the \n            prediction job in.\n        output_data_format (str): Optional. Format of the output data files, \n            defaults to JSON.\n        prediction_input (dict): Input parameters to create a prediction job.\n        job_id_prefix (str): the prefix of the generated job id.\n        wait_interval (int): optional wait interval between calls\n            to get job status. Defaults to 30.\n    \"\"\"\n    if not prediction_input:\n        prediction_input = {}\n    if not model_path:\n        raise ValueError('model_path must be provided.')\n    if _is_model_name(model_path):\n        prediction_input['modelName'] = model_path\n    elif _is_model_version_name(model_path):\n        prediction_input['versionName'] = model_path\n    elif _is_gcs_path(model_path):\n        prediction_input['uri'] = model_path\n    else:\n        raise ValueError('model_path value is invalid.')\n    \n    if input_paths:\n        prediction_input['inputPaths'] = input_paths\n    if input_data_format:\n        prediction_input['dataFormat'] = input_data_format\n    if output_path:\n        prediction_input['outputPath'] = output_path\n    if output_data_format:\n        prediction_input['outputDataFormat'] = output_data_format\n    if region:\n        prediction_input['region'] = region\n    job = {\n        'predictionInput': prediction_input\n    }\n    create_job(project_id, job, job_id_prefix, wait_interval, job_id_output_path=job_id_output_path)\n    \ndef _is_model_name(name):\n    return re.match(r'/projects/[^/]+/models/[^/]+$', name)\n\ndef _is_model_version_name(name):\n    return re.match(r'/projects/[^/]+/models/[^/]+/versions/[^/]+$', name)\n\ndef _is_gcs_path(name):\n    return name.startswith('gs://')", "path": "components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py"}], "after_files": [{"content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom fire import decorators\nfrom ._create_job import create_job\n\n@decorators.SetParseFns(python_version=str, runtime_version=str)\ndef train(project_id, job_id_output_path, python_module=None, package_uris=None, \n    region=None, args=None, job_dir=None, python_version=None, \n    runtime_version=None, master_image_uri=None, worker_image_uri=None, \n    training_input=None, job_id_prefix=None, job_id=None, wait_interval=30):\n    \"\"\"Creates a MLEngine training job.\n\n    Args:\n        project_id (str): Required. The ID of the parent project of the job.\n        python_module (str): Required. The Python module name to run after \n            installing the packages.\n        package_uris (list): Required. The Google Cloud Storage location of \n            the packages with the training program and any additional \n            dependencies. The maximum number of package URIs is 100.\n        region (str): Required. The Google Compute Engine region to run the \n            training job in\n        args (list): Command line arguments to pass to the program.\n        job_dir (str): A Google Cloud Storage path in which to store training \n            outputs and other data needed for training. This path is passed \n            to your TensorFlow program as the '--job-dir' command-line \n            argument. The benefit of specifying this field is that Cloud ML \n            validates the path for use in training.\n        python_version (str): Optional. The version of Python used in training. \n            If not set, the default version is '2.7'. Python '3.5' is \n            available when runtimeVersion is set to '1.4' and above. \n            Python '2.7' works with all supported runtime versions.\n        runtime_version (str): Optional. The Cloud ML Engine runtime version \n            to use for training. If not set, Cloud ML Engine uses the \n            default stable version, 1.0. \n        master_image_uri (str): The Docker image to run on the master replica. \n            This image must be in Container Registry.\n        worker_image_uri (str): The Docker image to run on the worker replica. \n            This image must be in Container Registry.\n        training_input (dict): Input parameters to create a training job.\n        job_id_prefix (str): the prefix of the generated job id.\n        job_id (str): the created job_id, takes precedence over generated job\n            id if set.\n        wait_interval (int): optional wait interval between calls\n            to get job status. Defaults to 30.\n    \"\"\"\n    if not training_input:\n        training_input = {}\n    if python_module:\n        training_input['pythonModule'] = python_module\n    if package_uris:\n        training_input['packageUris'] = package_uris\n    if region:\n        training_input['region'] = region\n    if args:\n        training_input['args'] = args\n    if job_dir:\n        training_input['jobDir'] = job_dir\n    if python_version:\n        training_input['pythonVersion'] = python_version\n    if runtime_version:\n        training_input['runtimeVersion'] = runtime_version\n    if master_image_uri:\n        if 'masterConfig' not in training_input:\n            training_input['masterConfig'] = {}\n        training_input['masterConfig']['imageUri'] = master_image_uri\n    if worker_image_uri:\n        if 'workerConfig' not in training_input:\n            training_input['workerConfig'] = {}\n        training_input['workerConfig']['imageUri'] = worker_image_uri\n    job = {\n        'trainingInput': training_input\n    }\n    return create_job(\n        project_id=project_id,\n        job=job,\n        job_id_prefix=job_id_prefix,\n        job_id=job_id,\n        wait_interval=wait_interval,\n        job_id_output_path=job_id_output_path,\n    )", "path": "components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_train.py"}, {"content": "# Copyright 2018 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport re\n\nfrom ._create_job import create_job\n\ndef batch_predict(project_id, model_path, input_paths, input_data_format, \n    output_path, region, job_id_output_path, output_data_format=None, prediction_input=None, job_id_prefix=None,\n    wait_interval=30):\n    \"\"\"Creates a MLEngine batch prediction job.\n\n    Args:\n        project_id (str): Required. The ID of the parent project of the job.\n        model_path (str): Required. The path to the model. It can be either:\n            `projects/[PROJECT_ID]/models/[MODEL_ID]` or \n            `projects/[PROJECT_ID]/models/[MODEL_ID]/versions/[VERSION_ID]`\n            or a GCS path of a model file.\n        input_paths (list): Required. The Google Cloud Storage location of \n            the input data files. May contain wildcards.\n        input_data_format (str): Required. The format of the input data files.\n            See https://cloud.google.com/ml-engine/reference/rest/v1/projects.jobs#DataFormat.\n        output_path (str): Required. The output Google Cloud Storage location.\n        region (str): Required. The Google Compute Engine region to run the \n            prediction job in.\n        output_data_format (str): Optional. Format of the output data files, \n            defaults to JSON.\n        prediction_input (dict): Input parameters to create a prediction job.\n        job_id_prefix (str): the prefix of the generated job id.\n        wait_interval (int): optional wait interval between calls\n            to get job status. Defaults to 30.\n    \"\"\"\n    if not prediction_input:\n        prediction_input = {}\n    if not model_path:\n        raise ValueError('model_path must be provided.')\n    if _is_model_name(model_path):\n        prediction_input['modelName'] = model_path\n    elif _is_model_version_name(model_path):\n        prediction_input['versionName'] = model_path\n    elif _is_gcs_path(model_path):\n        prediction_input['uri'] = model_path\n    else:\n        raise ValueError('model_path value is invalid.')\n    \n    if input_paths:\n        prediction_input['inputPaths'] = input_paths\n    if input_data_format:\n        prediction_input['dataFormat'] = input_data_format\n    if output_path:\n        prediction_input['outputPath'] = output_path\n    if output_data_format:\n        prediction_input['outputDataFormat'] = output_data_format\n    if region:\n        prediction_input['region'] = region\n    job = {\n        'predictionInput': prediction_input\n    }\n    create_job(\n        project_id=project_id,\n        job=job,\n        job_id_prefix=job_id_prefix,\n        wait_interval=wait_interval,\n        job_id_output_path=job_id_output_path,\n    )\n    \ndef _is_model_name(name):\n    return re.match(r'/projects/[^/]+/models/[^/]+$', name)\n\ndef _is_model_version_name(name):\n    return re.match(r'/projects/[^/]+/models/[^/]+/versions/[^/]+$', name)\n\ndef _is_gcs_path(name):\n    return name.startswith('gs://')", "path": "components/gcp/container/component_sdk/python/kfp_component/google/ml_engine/_batch_predict.py"}]}