We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
S3 storage-initializer, secret/annotations and AWS_REGION
/kind bug

**What steps did you take and what happened:**
Hi everybody! I am trying to deploy kfserving 0.6.0 standalone with the storage-initializer to fetch models from a local Swift cluster (exposing an S3 API compatible with v4 signature). I followed the [tutorial](https://github.com/kubeflow/kfserving/blob/master/docs/samples/storage/s3/README.md) and added the annotation for the `s3-region` as described (in my case, it needs to be `US` to avoid signature validation failures when contacting Swift). I checked after the k8s deploy that they were applied correctly to the container, and I found the following (as expected):

```
Init Containers:
  storage-initializer:
    [..]
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:     100m
      memory:  100Mi
    Environment:
      AWS_ACCESS_KEY_ID:      <set to the key 'AWS_ACCESS_KEY_ID' in secret 'test-secret'>      Optional: false
      AWS_SECRET_ACCESS_KEY:  <set to the key 'AWS_SECRET_ACCESS_KEY' in secret 'test-secret'>  Optional: false
      S3_USE_HTTPS:           1
      S3_ENDPOINT:            thanos-swift.discovery.wmnet
      AWS_ENDPOINT_URL:       https://thanos-swift.discovery.wmnet
      AWS_REGION:             US
    [..]
```
But the storage-initializer container still failed to fetch the model from Swift, due to the following error:

```
botocore.exceptions.ClientError: An error occurred (SignatureDoesNotMatch) when calling the ListObjects operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.
```
The above can be easily reproduced via a python-venv with boto3, testing something like (of course in this case it is my internal endpoint, to repro one needs to use another one like AWS etc..):
```
import boto3

# Retrieve the list of existing buckets
s3 = boto3.client('s3', aws_access_key_id="XXXXXXX", 
                  aws_secret_access_key="XXXXXXXX",
                  endpoint_url="https://thanos-swift.discovery.wmnet")
response = s3.list_buckets()
```
The above doesn't work if I export `AWS_REGION="US"`, but it does if I export `AWS_DEFAULT_REGION="US"`. This seems to be supported by https://github.com/boto/boto3/issues/2574 (if you check the attached pull request the code mentions the AWS_DEFAULT_REGION env variable only).

I am wondering if we should just sed/replace AWS_REGION with AWS_DEFAULT_REGION (given that we use boto) and possibly release 0.6.1, or if there is another workaround that I haven't checked. Happy to send a pull request in case, or happy to see others doing it, no preferences :)

**What did you expect to happen:**
I expected the storage-initializer to honor the AWS_REGION setting and fetch from Swift/S3 correctly.

**Anything else you would like to add:**
[Miscellaneous information that will assist in solving the issue.]


**Environment:**

- Istio Version: 1.9.5
- Knative Version: 0.18.1
- KFServing Version: 0.6.0
- Kubeflow version: none
- Kubernetes version: (use `kubectl version`): 1.16
- OS (e.g. from `/etc/os-release`): Debian 10 (Buster)


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `python/kfserving/kfserving/api/creds_utils.py`
Content:
```
1 # Copyright 2019 The Kubeflow Authors.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 
15 import logging
16 import json
17 import configparser
18 from os.path import expanduser
19 
20 from kubernetes import client
21 from ..constants import constants
22 
23 logger = logging.getLogger(__name__)
24 
25 
26 def set_gcs_credentials(namespace, credentials_file, service_account):
27     """
28     Set GCS Credentails (secret and service account) with credentials file.
29     Args:
30         namespace(str): The kubernetes namespace.
31         credentials_file(str): The path for the gcs credentials file.
32         service_account(str): The name of service account. If the service_account
33                               is specified, will attach created secret with the service account,
34                               otherwise will create new one and attach with created secret.
35     """
36 
37     with open(expanduser(credentials_file)) as f:
38         gcs_creds_content = f.read()
39 
40     # Try to get GCS creds file name from configmap, set default value then if cannot.
41     gcs_creds_file_name = get_creds_name_from_config_map(
42         'gcsCredentialFileName')
43     if not gcs_creds_file_name:
44         gcs_creds_file_name = constants.GCS_CREDS_FILE_DEFAULT_NAME
45 
46     string_data = {gcs_creds_file_name: gcs_creds_content}
47 
48     secret_name = create_secret(
49         namespace=namespace, string_data=string_data)
50 
51     set_service_account(namespace=namespace,
52                         service_account=service_account,
53                         secret_name=secret_name)
54 
55 
56 def set_s3_credentials(namespace, credentials_file, service_account,
57                        s3_profile='default',  # pylint: disable=too-many-locals,too-many-arguments
58                        s3_endpoint=None, s3_region=None, s3_use_https=None,
59                        s3_verify_ssl=None):  # pylint: disable=unused-argument
60     """
61     Set S3 Credentails (secret and service account).
62     Args:
63         namespace(str): The kubernetes namespace.
64         credentials_file(str): The path for the S3 credentials file.
65         s3_profile(str): The profile for S3, default value is 'default'.
66         service_account(str): The name of service account(Optional). If the service_account
67                               is specified, will attach created secret with the service account,
68                               otherwise will create new one and attach with created secret.
69         s3_endpoint(str): S3 settings variable S3_ENDPOINT.
70         s3_region(str): S3 settings variable AWS_REGION.
71         s3_use_https(str): S3 settings variable S3_USE_HTTPS.
72         s3_verify_ssl(str): S3 settings variable S3_VERIFY_SSL.
73     """
74 
75     config = configparser.ConfigParser()
76     config.read([expanduser(credentials_file)])
77     s3_access_key_id = config.get(s3_profile, 'aws_access_key_id')
78     s3_secret_access_key = config.get(
79         s3_profile, 'aws_secret_access_key')
80 
81     # Try to get S3 creds name from configmap, set default value then if cannot.
82     s3_access_key_id_name = get_creds_name_from_config_map(
83         's3AccessKeyIDName')
84     if not s3_access_key_id_name:
85         s3_access_key_id_name = constants.S3_ACCESS_KEY_ID_DEFAULT_NAME
86 
87     s3_secret_access_key_name = get_creds_name_from_config_map(
88         's3SecretAccessKeyName')
89     if not s3_secret_access_key_name:
90         s3_secret_access_key_name = constants.S3_SECRET_ACCESS_KEY_DEFAULT_NAME
91 
92     data = {
93         s3_access_key_id_name: s3_access_key_id,
94         s3_secret_access_key_name: s3_secret_access_key,
95     }
96 
97     s3_cred_sets = {
98         's3_endpoint': constants.KFSERVING_GROUP + "/s3-endpoint",
99         's3_region': constants.KFSERVING_GROUP + "/s3-region",
100         's3_use_https': constants.KFSERVING_GROUP + "/s3-usehttps",
101         's3_verify_ssl': constants.KFSERVING_GROUP + "/s3-verifyssl",
102     }
103 
104     s3_annotations = {}
105     for key, value in s3_cred_sets.items():
106         arg = vars()[key]
107         if arg is not None:
108             s3_annotations.update({value: arg})
109 
110     secret_name = create_secret(
111         namespace=namespace, annotations=s3_annotations, data=data)
112 
113     set_service_account(namespace=namespace,
114                         service_account=service_account,
115                         secret_name=secret_name)
116 
117 
118 def set_azure_credentials(namespace, credentials_file, service_account):
119     """
120     Set Azure Credentails (secret and service account) with credentials file.
121     Args:
122         namespace(str): The kubernetes namespace.
123         credentials_file(str): The path for the Azure credentials file.
124         service_account(str): The name of service account. If the service_account
125                               is specified, will attach created secret with the service account,
126                               otherwise will create new one and attach with created secret.
127     """
128 
129     with open(expanduser(credentials_file)) as azure_creds_file:
130         azure_creds = json.load(azure_creds_file)
131 
132     data = {
133         'AZ_CLIENT_ID': azure_creds['clientId'],
134         'AZ_CLIENT_SECRET': azure_creds['clientSecret'],
135         'AZ_SUBSCRIPTION_ID': azure_creds['subscriptionId'],
136         'AZ_TENANT_ID': azure_creds['tenantId'],
137         }
138 
139     secret_name = create_secret(
140         namespace=namespace, data=data)
141 
142     set_service_account(namespace=namespace,
143                         service_account=service_account,
144                         secret_name=secret_name)
145 
146 
147 def create_secret(namespace, annotations=None, data=None, string_data=None):
148     'Create namespaced secret, and return the secret name.'
149     try:
150         created_secret = client.CoreV1Api().create_namespaced_secret(
151             namespace,
152             client.V1Secret(
153                 api_version='v1',
154                 kind='Secret',
155                 metadata=client.V1ObjectMeta(
156                     generate_name=constants.DEFAULT_SECRET_NAME,
157                     annotations=annotations),
158                 data=data,
159                 string_data=string_data))
160     except client.rest.ApiException as e:
161         raise RuntimeError(
162             "Exception when calling CoreV1Api->create_namespaced_secret: %s\n" % e)
163 
164     secret_name = created_secret.metadata.name
165     logger.info('Created Secret: %s in namespace %s', secret_name, namespace)
166     return secret_name
167 
168 
169 def set_service_account(namespace, service_account, secret_name):
170     """
171     Set service account, create if service_account does not exist, otherwise patch it.
172     """
173     if check_sa_exists(namespace=namespace, service_account=service_account):
174         patch_service_account(secret_name=secret_name,
175                               namespace=namespace,
176                               sa_name=service_account)
177     else:
178         create_service_account(secret_name=secret_name,
179                                namespace=namespace,
180                                sa_name=service_account)
181 
182 
183 def check_sa_exists(namespace, service_account):
184     """
185     Check if the specified service account existing.
186     """
187     sa_list = client.CoreV1Api().list_namespaced_service_account(namespace=namespace)
188 
189     sa_name_list = [sa.metadata.name for sa in sa_list.items]
190 
191     if service_account in sa_name_list:
192         return True
193 
194     return False
195 
196 
197 def create_service_account(secret_name, namespace, sa_name):
198     """
199     Create namespaced service account, and return the service account name
200     """
201     try:
202         client.CoreV1Api().create_namespaced_service_account(
203             namespace,
204             client.V1ServiceAccount(
205                 metadata=client.V1ObjectMeta(
206                     name=sa_name
207                 ),
208                 secrets=[client.V1ObjectReference(
209                     kind='Secret',
210                     name=secret_name)]))
211     except client.rest.ApiException as e:
212         raise RuntimeError(
213             "Exception when calling CoreV1Api->create_namespaced_service_account: %s\n" % e)
214 
215     logger.info('Created Service account: %s in namespace %s', sa_name, namespace)
216 
217 
218 def patch_service_account(secret_name, namespace, sa_name):
219     """
220     Patch namespaced service account to attach with created secret.
221     """
222     try:
223         client.CoreV1Api().patch_namespaced_service_account(
224             sa_name,
225             namespace,
226             client.V1ServiceAccount(
227                 secrets=[client.V1ObjectReference(
228                     kind='Secret',
229                     name=secret_name)]))
230     except client.rest.ApiException as e:
231         raise RuntimeError(
232             "Exception when calling CoreV1Api->patch_namespaced_service_account: %s\n" % e)
233 
234     logger.info('Pacthed Service account: %s in namespace %s', sa_name, namespace)
235 
236 
237 def get_creds_name_from_config_map(creds):
238     """
239     Get the credentials name from inferenceservice config map.
240     """
241     try:
242         isvc_config_map = client.CoreV1Api().read_namespaced_config_map(
243             constants.INFERENCESERVICE_CONFIG_MAP_NAME,
244             constants.INFERENCESERVICE_SYSTEM_NAMESPACE)
245     except client.rest.ApiException:
246         logging.warning('Cannot get configmap %s in namespace %s.',
247                         constants.INFERENCESERVICE_CONFIG_MAP_NAME,
248                         constants.INFERENCESERVICE_SYSTEM_NAMESPACE)
249         return None
250 
251     isvc_creds_str = isvc_config_map.data['credentials']
252     isvc_creds_json = json.loads(isvc_creds_str)
253 
254     if creds == 'gcsCredentialFileName':
255         return isvc_creds_json['gcs']['gcsCredentialFileName']
256     elif creds == 's3AccessKeyIDName':
257         return isvc_creds_json['s3']['s3AccessKeyIDName']
258     elif creds == 's3SecretAccessKeyName':
259         return isvc_creds_json['s3']['s3SecretAccessKeyName']
260     else:
261         raise RuntimeError("Unknown credentials.")
262
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/python/kfserving/kfserving/api/creds_utils.py b/python/kfserving/kfserving/api/creds_utils.py
--- a/python/kfserving/kfserving/api/creds_utils.py
+++ b/python/kfserving/kfserving/api/creds_utils.py
@@ -56,7 +56,7 @@
 def set_s3_credentials(namespace, credentials_file, service_account,
                        s3_profile='default',  # pylint: disable=too-many-locals,too-many-arguments
                        s3_endpoint=None, s3_region=None, s3_use_https=None,
-                       s3_verify_ssl=None):  # pylint: disable=unused-argument
+                       s3_verify_ssl=None, s3_cabundle=None):  # pylint: disable=unused-argument
     """
     Set S3 Credentails (secret and service account).
     Args:
@@ -67,9 +67,10 @@
                               is specified, will attach created secret with the service account,
                               otherwise will create new one and attach with created secret.
         s3_endpoint(str): S3 settings variable S3_ENDPOINT.
-        s3_region(str): S3 settings variable AWS_REGION.
+        s3_region(str): S3 settings variable AWS_DEFAULT_REGION.
         s3_use_https(str): S3 settings variable S3_USE_HTTPS.
         s3_verify_ssl(str): S3 settings variable S3_VERIFY_SSL.
+        s3_cabundle(str): S3 settings variable AWS_CA_BUNDLE.
     """
 
     config = configparser.ConfigParser()
@@ -99,6 +100,7 @@
         's3_region': constants.KFSERVING_GROUP + "/s3-region",
         's3_use_https': constants.KFSERVING_GROUP + "/s3-usehttps",
         's3_verify_ssl': constants.KFSERVING_GROUP + "/s3-verifyssl",
+        's3_cabundle': constants.KFSERVING_GROUP + "/s3-cabundle",
     }
 
     s3_annotations = {}

{"golden_diff": "diff --git a/python/kfserving/kfserving/api/creds_utils.py b/python/kfserving/kfserving/api/creds_utils.py\n--- a/python/kfserving/kfserving/api/creds_utils.py\n+++ b/python/kfserving/kfserving/api/creds_utils.py\n@@ -56,7 +56,7 @@\n def set_s3_credentials(namespace, credentials_file, service_account,\n                        s3_profile='default',  # pylint: disable=too-many-locals,too-many-arguments\n                        s3_endpoint=None, s3_region=None, s3_use_https=None,\n-                       s3_verify_ssl=None):  # pylint: disable=unused-argument\n+                       s3_verify_ssl=None, s3_cabundle=None):  # pylint: disable=unused-argument\n     \"\"\"\n     Set S3 Credentails (secret and service account).\n     Args:\n@@ -67,9 +67,10 @@\n                               is specified, will attach created secret with the service account,\n                               otherwise will create new one and attach with created secret.\n         s3_endpoint(str): S3 settings variable S3_ENDPOINT.\n-        s3_region(str): S3 settings variable AWS_REGION.\n+        s3_region(str): S3 settings variable AWS_DEFAULT_REGION.\n         s3_use_https(str): S3 settings variable S3_USE_HTTPS.\n         s3_verify_ssl(str): S3 settings variable S3_VERIFY_SSL.\n+        s3_cabundle(str): S3 settings variable AWS_CA_BUNDLE.\n     \"\"\"\n \n     config = configparser.ConfigParser()\n@@ -99,6 +100,7 @@\n         's3_region': constants.KFSERVING_GROUP + \"/s3-region\",\n         's3_use_https': constants.KFSERVING_GROUP + \"/s3-usehttps\",\n         's3_verify_ssl': constants.KFSERVING_GROUP + \"/s3-verifyssl\",\n+        's3_cabundle': constants.KFSERVING_GROUP + \"/s3-cabundle\",\n     }\n \n     s3_annotations = {}\n", "issue": "S3 storage-initializer, secret/annotations and AWS_REGION\n/kind bug\r\n\r\n**What steps did you take and what happened:**\r\nHi everybody! I am trying to deploy kfserving 0.6.0 standalone with the storage-initializer to fetch models from a local Swift cluster (exposing an S3 API compatible with v4 signature). I followed the [tutorial](https://github.com/kubeflow/kfserving/blob/master/docs/samples/storage/s3/README.md) and added the annotation for the `s3-region` as described (in my case, it needs to be `US` to avoid signature validation failures when contacting Swift). I checked after the k8s deploy that they were applied correctly to the container, and I found the following (as expected):\r\n\r\n```\r\nInit Containers:\r\n  storage-initializer:\r\n    [..]\r\n    Limits:\r\n      cpu:     1\r\n      memory:  1Gi\r\n    Requests:\r\n      cpu:     100m\r\n      memory:  100Mi\r\n    Environment:\r\n      AWS_ACCESS_KEY_ID:      <set to the key 'AWS_ACCESS_KEY_ID' in secret 'test-secret'>      Optional: false\r\n      AWS_SECRET_ACCESS_KEY:  <set to the key 'AWS_SECRET_ACCESS_KEY' in secret 'test-secret'>  Optional: false\r\n      S3_USE_HTTPS:           1\r\n      S3_ENDPOINT:            thanos-swift.discovery.wmnet\r\n      AWS_ENDPOINT_URL:       https://thanos-swift.discovery.wmnet\r\n      AWS_REGION:             US\r\n    [..]\r\n```\r\nBut the storage-initializer container still failed to fetch the model from Swift, due to the following error:\r\n\r\n```\r\nbotocore.exceptions.ClientError: An error occurred (SignatureDoesNotMatch) when calling the ListObjects operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.\r\n```\r\nThe above can be easily reproduced via a python-venv with boto3, testing something like (of course in this case it is my internal endpoint, to repro one needs to use another one like AWS etc..):\r\n```\r\nimport boto3\r\n\r\n# Retrieve the list of existing buckets\r\ns3 = boto3.client('s3', aws_access_key_id=\"XXXXXXX\", \r\n                  aws_secret_access_key=\"XXXXXXXX\",\r\n                  endpoint_url=\"https://thanos-swift.discovery.wmnet\")\r\nresponse = s3.list_buckets()\r\n```\r\nThe above doesn't work if I export `AWS_REGION=\"US\"`, but it does if I export `AWS_DEFAULT_REGION=\"US\"`. This seems to be supported by https://github.com/boto/boto3/issues/2574 (if you check the attached pull request the code mentions the AWS_DEFAULT_REGION env variable only).\r\n\r\nI am wondering if we should just sed/replace AWS_REGION with AWS_DEFAULT_REGION (given that we use boto) and possibly release 0.6.1, or if there is another workaround that I haven't checked. Happy to send a pull request in case, or happy to see others doing it, no preferences :)\r\n\r\n**What did you expect to happen:**\r\nI expected the storage-initializer to honor the AWS_REGION setting and fetch from Swift/S3 correctly.\r\n\r\n**Anything else you would like to add:**\r\n[Miscellaneous information that will assist in solving the issue.]\r\n\r\n\r\n**Environment:**\r\n\r\n- Istio Version: 1.9.5\r\n- Knative Version: 0.18.1\r\n- KFServing Version: 0.6.0\r\n- Kubeflow version: none\r\n- Kubernetes version: (use `kubectl version`): 1.16\r\n- OS (e.g. from `/etc/os-release`): Debian 10 (Buster)\r\n\n", "before_files": [{"content": "# Copyright 2019 The Kubeflow Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport logging\nimport json\nimport configparser\nfrom os.path import expanduser\n\nfrom kubernetes import client\nfrom ..constants import constants\n\nlogger = logging.getLogger(__name__)\n\n\ndef set_gcs_credentials(namespace, credentials_file, service_account):\n    \"\"\"\n    Set GCS Credentails (secret and service account) with credentials file.\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the gcs credentials file.\n        service_account(str): The name of service account. If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n    \"\"\"\n\n    with open(expanduser(credentials_file)) as f:\n        gcs_creds_content = f.read()\n\n    # Try to get GCS creds file name from configmap, set default value then if cannot.\n    gcs_creds_file_name = get_creds_name_from_config_map(\n        'gcsCredentialFileName')\n    if not gcs_creds_file_name:\n        gcs_creds_file_name = constants.GCS_CREDS_FILE_DEFAULT_NAME\n\n    string_data = {gcs_creds_file_name: gcs_creds_content}\n\n    secret_name = create_secret(\n        namespace=namespace, string_data=string_data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef set_s3_credentials(namespace, credentials_file, service_account,\n                       s3_profile='default',  # pylint: disable=too-many-locals,too-many-arguments\n                       s3_endpoint=None, s3_region=None, s3_use_https=None,\n                       s3_verify_ssl=None):  # pylint: disable=unused-argument\n    \"\"\"\n    Set S3 Credentails (secret and service account).\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the S3 credentials file.\n        s3_profile(str): The profile for S3, default value is 'default'.\n        service_account(str): The name of service account(Optional). If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n        s3_endpoint(str): S3 settings variable S3_ENDPOINT.\n        s3_region(str): S3 settings variable AWS_REGION.\n        s3_use_https(str): S3 settings variable S3_USE_HTTPS.\n        s3_verify_ssl(str): S3 settings variable S3_VERIFY_SSL.\n    \"\"\"\n\n    config = configparser.ConfigParser()\n    config.read([expanduser(credentials_file)])\n    s3_access_key_id = config.get(s3_profile, 'aws_access_key_id')\n    s3_secret_access_key = config.get(\n        s3_profile, 'aws_secret_access_key')\n\n    # Try to get S3 creds name from configmap, set default value then if cannot.\n    s3_access_key_id_name = get_creds_name_from_config_map(\n        's3AccessKeyIDName')\n    if not s3_access_key_id_name:\n        s3_access_key_id_name = constants.S3_ACCESS_KEY_ID_DEFAULT_NAME\n\n    s3_secret_access_key_name = get_creds_name_from_config_map(\n        's3SecretAccessKeyName')\n    if not s3_secret_access_key_name:\n        s3_secret_access_key_name = constants.S3_SECRET_ACCESS_KEY_DEFAULT_NAME\n\n    data = {\n        s3_access_key_id_name: s3_access_key_id,\n        s3_secret_access_key_name: s3_secret_access_key,\n    }\n\n    s3_cred_sets = {\n        's3_endpoint': constants.KFSERVING_GROUP + \"/s3-endpoint\",\n        's3_region': constants.KFSERVING_GROUP + \"/s3-region\",\n        's3_use_https': constants.KFSERVING_GROUP + \"/s3-usehttps\",\n        's3_verify_ssl': constants.KFSERVING_GROUP + \"/s3-verifyssl\",\n    }\n\n    s3_annotations = {}\n    for key, value in s3_cred_sets.items():\n        arg = vars()[key]\n        if arg is not None:\n            s3_annotations.update({value: arg})\n\n    secret_name = create_secret(\n        namespace=namespace, annotations=s3_annotations, data=data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef set_azure_credentials(namespace, credentials_file, service_account):\n    \"\"\"\n    Set Azure Credentails (secret and service account) with credentials file.\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the Azure credentials file.\n        service_account(str): The name of service account. If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n    \"\"\"\n\n    with open(expanduser(credentials_file)) as azure_creds_file:\n        azure_creds = json.load(azure_creds_file)\n\n    data = {\n        'AZ_CLIENT_ID': azure_creds['clientId'],\n        'AZ_CLIENT_SECRET': azure_creds['clientSecret'],\n        'AZ_SUBSCRIPTION_ID': azure_creds['subscriptionId'],\n        'AZ_TENANT_ID': azure_creds['tenantId'],\n        }\n\n    secret_name = create_secret(\n        namespace=namespace, data=data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef create_secret(namespace, annotations=None, data=None, string_data=None):\n    'Create namespaced secret, and return the secret name.'\n    try:\n        created_secret = client.CoreV1Api().create_namespaced_secret(\n            namespace,\n            client.V1Secret(\n                api_version='v1',\n                kind='Secret',\n                metadata=client.V1ObjectMeta(\n                    generate_name=constants.DEFAULT_SECRET_NAME,\n                    annotations=annotations),\n                data=data,\n                string_data=string_data))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->create_namespaced_secret: %s\\n\" % e)\n\n    secret_name = created_secret.metadata.name\n    logger.info('Created Secret: %s in namespace %s', secret_name, namespace)\n    return secret_name\n\n\ndef set_service_account(namespace, service_account, secret_name):\n    \"\"\"\n    Set service account, create if service_account does not exist, otherwise patch it.\n    \"\"\"\n    if check_sa_exists(namespace=namespace, service_account=service_account):\n        patch_service_account(secret_name=secret_name,\n                              namespace=namespace,\n                              sa_name=service_account)\n    else:\n        create_service_account(secret_name=secret_name,\n                               namespace=namespace,\n                               sa_name=service_account)\n\n\ndef check_sa_exists(namespace, service_account):\n    \"\"\"\n    Check if the specified service account existing.\n    \"\"\"\n    sa_list = client.CoreV1Api().list_namespaced_service_account(namespace=namespace)\n\n    sa_name_list = [sa.metadata.name for sa in sa_list.items]\n\n    if service_account in sa_name_list:\n        return True\n\n    return False\n\n\ndef create_service_account(secret_name, namespace, sa_name):\n    \"\"\"\n    Create namespaced service account, and return the service account name\n    \"\"\"\n    try:\n        client.CoreV1Api().create_namespaced_service_account(\n            namespace,\n            client.V1ServiceAccount(\n                metadata=client.V1ObjectMeta(\n                    name=sa_name\n                ),\n                secrets=[client.V1ObjectReference(\n                    kind='Secret',\n                    name=secret_name)]))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->create_namespaced_service_account: %s\\n\" % e)\n\n    logger.info('Created Service account: %s in namespace %s', sa_name, namespace)\n\n\ndef patch_service_account(secret_name, namespace, sa_name):\n    \"\"\"\n    Patch namespaced service account to attach with created secret.\n    \"\"\"\n    try:\n        client.CoreV1Api().patch_namespaced_service_account(\n            sa_name,\n            namespace,\n            client.V1ServiceAccount(\n                secrets=[client.V1ObjectReference(\n                    kind='Secret',\n                    name=secret_name)]))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->patch_namespaced_service_account: %s\\n\" % e)\n\n    logger.info('Pacthed Service account: %s in namespace %s', sa_name, namespace)\n\n\ndef get_creds_name_from_config_map(creds):\n    \"\"\"\n    Get the credentials name from inferenceservice config map.\n    \"\"\"\n    try:\n        isvc_config_map = client.CoreV1Api().read_namespaced_config_map(\n            constants.INFERENCESERVICE_CONFIG_MAP_NAME,\n            constants.INFERENCESERVICE_SYSTEM_NAMESPACE)\n    except client.rest.ApiException:\n        logging.warning('Cannot get configmap %s in namespace %s.',\n                        constants.INFERENCESERVICE_CONFIG_MAP_NAME,\n                        constants.INFERENCESERVICE_SYSTEM_NAMESPACE)\n        return None\n\n    isvc_creds_str = isvc_config_map.data['credentials']\n    isvc_creds_json = json.loads(isvc_creds_str)\n\n    if creds == 'gcsCredentialFileName':\n        return isvc_creds_json['gcs']['gcsCredentialFileName']\n    elif creds == 's3AccessKeyIDName':\n        return isvc_creds_json['s3']['s3AccessKeyIDName']\n    elif creds == 's3SecretAccessKeyName':\n        return isvc_creds_json['s3']['s3SecretAccessKeyName']\n    else:\n        raise RuntimeError(\"Unknown credentials.\")\n", "path": "python/kfserving/kfserving/api/creds_utils.py"}], "after_files": [{"content": "# Copyright 2019 The Kubeflow Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport logging\nimport json\nimport configparser\nfrom os.path import expanduser\n\nfrom kubernetes import client\nfrom ..constants import constants\n\nlogger = logging.getLogger(__name__)\n\n\ndef set_gcs_credentials(namespace, credentials_file, service_account):\n    \"\"\"\n    Set GCS Credentails (secret and service account) with credentials file.\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the gcs credentials file.\n        service_account(str): The name of service account. If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n    \"\"\"\n\n    with open(expanduser(credentials_file)) as f:\n        gcs_creds_content = f.read()\n\n    # Try to get GCS creds file name from configmap, set default value then if cannot.\n    gcs_creds_file_name = get_creds_name_from_config_map(\n        'gcsCredentialFileName')\n    if not gcs_creds_file_name:\n        gcs_creds_file_name = constants.GCS_CREDS_FILE_DEFAULT_NAME\n\n    string_data = {gcs_creds_file_name: gcs_creds_content}\n\n    secret_name = create_secret(\n        namespace=namespace, string_data=string_data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef set_s3_credentials(namespace, credentials_file, service_account,\n                       s3_profile='default',  # pylint: disable=too-many-locals,too-many-arguments\n                       s3_endpoint=None, s3_region=None, s3_use_https=None,\n                       s3_verify_ssl=None, s3_cabundle=None):  # pylint: disable=unused-argument\n    \"\"\"\n    Set S3 Credentails (secret and service account).\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the S3 credentials file.\n        s3_profile(str): The profile for S3, default value is 'default'.\n        service_account(str): The name of service account(Optional). If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n        s3_endpoint(str): S3 settings variable S3_ENDPOINT.\n        s3_region(str): S3 settings variable AWS_DEFAULT_REGION.\n        s3_use_https(str): S3 settings variable S3_USE_HTTPS.\n        s3_verify_ssl(str): S3 settings variable S3_VERIFY_SSL.\n        s3_cabundle(str): S3 settings variable AWS_CA_BUNDLE.\n    \"\"\"\n\n    config = configparser.ConfigParser()\n    config.read([expanduser(credentials_file)])\n    s3_access_key_id = config.get(s3_profile, 'aws_access_key_id')\n    s3_secret_access_key = config.get(\n        s3_profile, 'aws_secret_access_key')\n\n    # Try to get S3 creds name from configmap, set default value then if cannot.\n    s3_access_key_id_name = get_creds_name_from_config_map(\n        's3AccessKeyIDName')\n    if not s3_access_key_id_name:\n        s3_access_key_id_name = constants.S3_ACCESS_KEY_ID_DEFAULT_NAME\n\n    s3_secret_access_key_name = get_creds_name_from_config_map(\n        's3SecretAccessKeyName')\n    if not s3_secret_access_key_name:\n        s3_secret_access_key_name = constants.S3_SECRET_ACCESS_KEY_DEFAULT_NAME\n\n    data = {\n        s3_access_key_id_name: s3_access_key_id,\n        s3_secret_access_key_name: s3_secret_access_key,\n    }\n\n    s3_cred_sets = {\n        's3_endpoint': constants.KFSERVING_GROUP + \"/s3-endpoint\",\n        's3_region': constants.KFSERVING_GROUP + \"/s3-region\",\n        's3_use_https': constants.KFSERVING_GROUP + \"/s3-usehttps\",\n        's3_verify_ssl': constants.KFSERVING_GROUP + \"/s3-verifyssl\",\n        's3_cabundle': constants.KFSERVING_GROUP + \"/s3-cabundle\",\n    }\n\n    s3_annotations = {}\n    for key, value in s3_cred_sets.items():\n        arg = vars()[key]\n        if arg is not None:\n            s3_annotations.update({value: arg})\n\n    secret_name = create_secret(\n        namespace=namespace, annotations=s3_annotations, data=data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef set_azure_credentials(namespace, credentials_file, service_account):\n    \"\"\"\n    Set Azure Credentails (secret and service account) with credentials file.\n    Args:\n        namespace(str): The kubernetes namespace.\n        credentials_file(str): The path for the Azure credentials file.\n        service_account(str): The name of service account. If the service_account\n                              is specified, will attach created secret with the service account,\n                              otherwise will create new one and attach with created secret.\n    \"\"\"\n\n    with open(expanduser(credentials_file)) as azure_creds_file:\n        azure_creds = json.load(azure_creds_file)\n\n    data = {\n        'AZ_CLIENT_ID': azure_creds['clientId'],\n        'AZ_CLIENT_SECRET': azure_creds['clientSecret'],\n        'AZ_SUBSCRIPTION_ID': azure_creds['subscriptionId'],\n        'AZ_TENANT_ID': azure_creds['tenantId'],\n        }\n\n    secret_name = create_secret(\n        namespace=namespace, data=data)\n\n    set_service_account(namespace=namespace,\n                        service_account=service_account,\n                        secret_name=secret_name)\n\n\ndef create_secret(namespace, annotations=None, data=None, string_data=None):\n    'Create namespaced secret, and return the secret name.'\n    try:\n        created_secret = client.CoreV1Api().create_namespaced_secret(\n            namespace,\n            client.V1Secret(\n                api_version='v1',\n                kind='Secret',\n                metadata=client.V1ObjectMeta(\n                    generate_name=constants.DEFAULT_SECRET_NAME,\n                    annotations=annotations),\n                data=data,\n                string_data=string_data))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->create_namespaced_secret: %s\\n\" % e)\n\n    secret_name = created_secret.metadata.name\n    logger.info('Created Secret: %s in namespace %s', secret_name, namespace)\n    return secret_name\n\n\ndef set_service_account(namespace, service_account, secret_name):\n    \"\"\"\n    Set service account, create if service_account does not exist, otherwise patch it.\n    \"\"\"\n    if check_sa_exists(namespace=namespace, service_account=service_account):\n        patch_service_account(secret_name=secret_name,\n                              namespace=namespace,\n                              sa_name=service_account)\n    else:\n        create_service_account(secret_name=secret_name,\n                               namespace=namespace,\n                               sa_name=service_account)\n\n\ndef check_sa_exists(namespace, service_account):\n    \"\"\"\n    Check if the specified service account existing.\n    \"\"\"\n    sa_list = client.CoreV1Api().list_namespaced_service_account(namespace=namespace)\n\n    sa_name_list = [sa.metadata.name for sa in sa_list.items]\n\n    if service_account in sa_name_list:\n        return True\n\n    return False\n\n\ndef create_service_account(secret_name, namespace, sa_name):\n    \"\"\"\n    Create namespaced service account, and return the service account name\n    \"\"\"\n    try:\n        client.CoreV1Api().create_namespaced_service_account(\n            namespace,\n            client.V1ServiceAccount(\n                metadata=client.V1ObjectMeta(\n                    name=sa_name\n                ),\n                secrets=[client.V1ObjectReference(\n                    kind='Secret',\n                    name=secret_name)]))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->create_namespaced_service_account: %s\\n\" % e)\n\n    logger.info('Created Service account: %s in namespace %s', sa_name, namespace)\n\n\ndef patch_service_account(secret_name, namespace, sa_name):\n    \"\"\"\n    Patch namespaced service account to attach with created secret.\n    \"\"\"\n    try:\n        client.CoreV1Api().patch_namespaced_service_account(\n            sa_name,\n            namespace,\n            client.V1ServiceAccount(\n                secrets=[client.V1ObjectReference(\n                    kind='Secret',\n                    name=secret_name)]))\n    except client.rest.ApiException as e:\n        raise RuntimeError(\n            \"Exception when calling CoreV1Api->patch_namespaced_service_account: %s\\n\" % e)\n\n    logger.info('Pacthed Service account: %s in namespace %s', sa_name, namespace)\n\n\ndef get_creds_name_from_config_map(creds):\n    \"\"\"\n    Get the credentials name from inferenceservice config map.\n    \"\"\"\n    try:\n        isvc_config_map = client.CoreV1Api().read_namespaced_config_map(\n            constants.INFERENCESERVICE_CONFIG_MAP_NAME,\n            constants.INFERENCESERVICE_SYSTEM_NAMESPACE)\n    except client.rest.ApiException:\n        logging.warning('Cannot get configmap %s in namespace %s.',\n                        constants.INFERENCESERVICE_CONFIG_MAP_NAME,\n                        constants.INFERENCESERVICE_SYSTEM_NAMESPACE)\n        return None\n\n    isvc_creds_str = isvc_config_map.data['credentials']\n    isvc_creds_json = json.loads(isvc_creds_str)\n\n    if creds == 'gcsCredentialFileName':\n        return isvc_creds_json['gcs']['gcsCredentialFileName']\n    elif creds == 's3AccessKeyIDName':\n        return isvc_creds_json['s3']['s3AccessKeyIDName']\n    elif creds == 's3SecretAccessKeyName':\n        return isvc_creds_json['s3']['s3SecretAccessKeyName']\n    else:\n        raise RuntimeError(\"Unknown credentials.\")\n", "path": "python/kfserving/kfserving/api/creds_utils.py"}]}