We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Dynamic Configuration values don't work with nested objects
I use the following cmd to set a key in the datastore.
`$ st2 key set -e smtp_account.me.password "I'm Encrypted"`

and here are my email pack config /opt/stackstorm/configs/email.yaml

```yaml
---
smtp_accounts:
  - name: name
    password: "{{ st2kv.system.smtp_account.me.password }}"
```
(followed by the rest of the account configuration).

This is based on the email pack's `config.schema.yaml`, which contains this:

```yaml
imap_accounts:
  description: "IMAP accounts"
  type: "array"
  required: true
  items:
    type: "object"
    required: true
    properties:
      name:
        description: "Name of the account"
        type: "string"
        secret: false
        required: true
      server:
        description: "Email server name - e.g. imap.gmail.com"
        type: "string"
        secret: false
        required: true
      username:
        description: "Mailbox username"
        type: "string"
        required: true
      password:
        description: "Mailbox password."
        type: "string"
        secret: true
        required: true
```

Running the email action fails:
```
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 278, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 171, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 49, in run
    s.login(account_data['username'], account_data['password'])
  File "/usr/lib64/python2.7/smtplib.py", line 621, in login
    raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (460, 'ERR.LOGIN.PASSERR')
```

I added some log to see the password value in File "/opt/stackstorm/packs/email/actions/send_email.py", line 49.
It seems that we get
`{{ st2kv.system.smtp_account.me.password }}` as the password.

FYI
If I run the send email action and use `{{ st2kv.system.smtp_account.me.password }}` as the message
I can see the message's value is correct.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `st2common/st2common/util/config_loader.py`
Content:
```
1 # Licensed to the StackStorm, Inc ('StackStorm') under one or more
2 # contributor license agreements.  See the NOTICE file distributed with
3 # this work for additional information regarding copyright ownership.
4 # The ASF licenses this file to You under the Apache License, Version 2.0
5 # (the "License"); you may not use this file except in compliance with
6 # the License.  You may obtain a copy of the License at
7 #
8 #     http://www.apache.org/licenses/LICENSE-2.0
9 #
10 # Unless required by applicable law or agreed to in writing, software
11 # distributed under the License is distributed on an "AS IS" BASIS,
12 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 # See the License for the specific language governing permissions and
14 # limitations under the License.
15 
16 from __future__ import absolute_import
17 import copy
18 
19 import six
20 
21 from oslo_config import cfg
22 
23 from st2common import log as logging
24 from st2common.models.db.pack import ConfigDB
25 from st2common.persistence.pack import ConfigSchema
26 from st2common.persistence.pack import Config
27 from st2common.content import utils as content_utils
28 from st2common.util import jinja as jinja_utils
29 from st2common.util.templating import render_template_with_system_and_user_context
30 from st2common.util.config_parser import ContentPackConfigParser
31 from st2common.exceptions.db import StackStormDBObjectNotFoundError
32 
33 __all__ = [
34     'ContentPackConfigLoader'
35 ]
36 
37 LOG = logging.getLogger(__name__)
38 
39 
40 class ContentPackConfigLoader(object):
41     """
42     Class which loads and resolves all the config values and returns a dictionary of resolved values
43     which can be passed to the resource.
44 
45     It loads and resolves values in the following order:
46 
47     1. Static values from <pack path>/config.yaml file
48     2. Dynamic and or static values from /opt/stackstorm/configs/<pack name>.yaml file.
49 
50     Values are merged from left to right which means values from "<pack name>.yaml" file have
51     precedence and override values from pack local config file.
52     """
53 
54     def __init__(self, pack_name, user=None):
55         self.pack_name = pack_name
56         self.user = user or cfg.CONF.system_user.user
57 
58         self.pack_path = content_utils.get_pack_base_path(pack_name=pack_name)
59         self._config_parser = ContentPackConfigParser(pack_name=pack_name)
60 
61     def get_config(self):
62         result = {}
63 
64         # Retrieve corresponding ConfigDB and ConfigSchemaDB object
65         # Note: ConfigSchemaDB is optional right now. If it doesn't exist, we assume every value
66         # is of a type string
67         try:
68             config_db = Config.get_by_pack(value=self.pack_name)
69         except StackStormDBObjectNotFoundError:
70             # Corresponding pack config doesn't exist. We set config_db to an empty config so
71             # that the default values from config schema are still correctly applied even if
72             # pack doesn't contain a config.
73             config_db = ConfigDB(pack=self.pack_name, values={})
74 
75         try:
76             config_schema_db = ConfigSchema.get_by_pack(value=self.pack_name)
77         except StackStormDBObjectNotFoundError:
78             config_schema_db = None
79 
80         # 2. Retrieve values from "global" pack config file (if available) and resolve them if
81         # necessary
82         config = self._get_values_for_config(config_schema_db=config_schema_db,
83                                              config_db=config_db)
84         result.update(config)
85 
86         return result
87 
88     def _get_values_for_config(self, config_schema_db, config_db):
89         schema_values = getattr(config_schema_db, 'attributes', {})
90         config_values = getattr(config_db, 'values', {})
91 
92         config = copy.deepcopy(config_values)
93 
94         # Assign dynamic config values based on the values in the datastore
95         config = self._assign_dynamic_config_values(schema=schema_values, config=config)
96 
97         # If config_schema is available we do a second pass and set default values for required
98         # items which values are not provided / available in the config itself
99         config = self._assign_default_values(schema=schema_values, config=config)
100         return config
101 
102     def _assign_dynamic_config_values(self, schema, config, parent_keys=None):
103         """
104         Assign dynamic config value for a particular config item if the ite utilizes a Jinja
105         expression for dynamic config values.
106 
107         Note: This method mutates config argument in place.
108 
109         :rtype: ``dict``
110         """
111         parent_keys = parent_keys or []
112 
113         for config_item_key, config_item_value in six.iteritems(config):
114             schema_item = schema.get(config_item_key, {})
115             is_dictionary = isinstance(config_item_value, dict)
116 
117             # Inspect nested object properties
118             if is_dictionary:
119                 parent_keys += [config_item_key]
120                 self._assign_dynamic_config_values(schema=schema_item.get('properties', {}),
121                                                    config=config[config_item_key],
122                                                    parent_keys=parent_keys)
123             else:
124                 is_jinja_expression = jinja_utils.is_jinja_expression(value=config_item_value)
125 
126                 if is_jinja_expression:
127                     # Resolve / render the Jinja template expression
128                     full_config_item_key = '.'.join(parent_keys + [config_item_key])
129                     value = self._get_datastore_value_for_expression(key=full_config_item_key,
130                         value=config_item_value,
131                         config_schema_item=schema_item)
132 
133                     config[config_item_key] = value
134                 else:
135                     # Static value, no resolution needed
136                     config[config_item_key] = config_item_value
137 
138         return config
139 
140     def _assign_default_values(self, schema, config):
141         """
142         Assign default values for particular config if default values are provided in the config
143         schema and a value is not specified in the config.
144 
145         Note: This method mutates config argument in place.
146 
147         :rtype: ``dict``
148         """
149         for schema_item_key, schema_item in six.iteritems(schema):
150             has_default_value = 'default' in schema_item
151             has_config_value = schema_item_key in config
152 
153             default_value = schema_item.get('default', None)
154             is_object = schema_item.get('type', None) == 'object'
155             has_properties = schema_item.get('properties', None)
156 
157             if has_default_value and not has_config_value:
158                 # Config value is not provided, but default value is, use a default value
159                 config[schema_item_key] = default_value
160 
161             # Inspect nested object properties
162             if is_object and has_properties:
163                 if not config.get(schema_item_key, None):
164                     config[schema_item_key] = {}
165 
166                 self._assign_default_values(schema=schema_item['properties'],
167                                             config=config[schema_item_key])
168 
169         return config
170 
171     def _get_datastore_value_for_expression(self, key, value, config_schema_item=None):
172         """
173         Retrieve datastore value by first resolving the datastore expression and then retrieving
174         the value from the datastore.
175 
176         :param key: Full path to the config item key (e.g. "token" / "auth.settings.token", etc.)
177         """
178         from st2common.services.config import deserialize_key_value
179 
180         config_schema_item = config_schema_item or {}
181         secret = config_schema_item.get('secret', False)
182 
183         try:
184             value = render_template_with_system_and_user_context(value=value,
185                                                                  user=self.user)
186         except Exception as e:
187             # Throw a more user-friendly exception on failed render
188             exc_class = type(e)
189             original_msg = str(e)
190             msg = ('Failed to render dynamic configuration value for key "%s" with value '
191                    '"%s" for pack "%s" config: %s ' % (key, value, self.pack_name, original_msg))
192             raise exc_class(msg)
193 
194         if value:
195             # Deserialize the value
196             value = deserialize_key_value(value=value, secret=secret)
197         else:
198             value = None
199 
200         return value
201 
202 
203 def get_config(pack, user):
204     """Returns config for given pack and user.
205     """
206     LOG.debug('Attempting to get config')
207     if pack and user:
208         LOG.debug('Pack and user found. Loading config.')
209         config_loader = ContentPackConfigLoader(
210             pack_name=pack,
211             user=user
212         )
213 
214         config = config_loader.get_config()
215 
216     else:
217         config = {}
218 
219     LOG.debug('Config: %s', config)
220 
221     return config
222
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/st2common/st2common/util/config_loader.py b/st2common/st2common/util/config_loader.py
--- a/st2common/st2common/util/config_loader.py
+++ b/st2common/st2common/util/config_loader.py
@@ -110,22 +110,42 @@
         """
         parent_keys = parent_keys or []
 
-        for config_item_key, config_item_value in six.iteritems(config):
-            schema_item = schema.get(config_item_key, {})
+        config_is_dict = isinstance(config, dict)
+        config_is_list = isinstance(config, list)
+        iterator = six.iteritems(config) if config_is_dict else enumerate(config)
+
+        # config_item_key - if config_is_dict then this is the key in the dictionary
+        #                   if config_is_list then this is the index of them item
+        # config_item_value - the value of the key/index for the current item
+        for config_item_key, config_item_value in iterator:
+            if config_is_dict:
+                # different schema for each key/value pair
+                schema_item = schema.get(config_item_key, {})
+            if config_is_list:
+                # same schema is shared between every item in the list
+                schema_item = schema
+
             is_dictionary = isinstance(config_item_value, dict)
+            is_list = isinstance(config_item_value, list)
 
             # Inspect nested object properties
             if is_dictionary:
-                parent_keys += [config_item_key]
+                parent_keys += [str(config_item_key)]
                 self._assign_dynamic_config_values(schema=schema_item.get('properties', {}),
                                                    config=config[config_item_key],
                                                    parent_keys=parent_keys)
+            # Inspect nested list items
+            elif is_list:
+                parent_keys += [str(config_item_key)]
+                self._assign_dynamic_config_values(schema=schema_item.get('items', {}),
+                                                   config=config[config_item_key],
+                                                   parent_keys=parent_keys)
             else:
                 is_jinja_expression = jinja_utils.is_jinja_expression(value=config_item_value)
 
                 if is_jinja_expression:
                     # Resolve / render the Jinja template expression
-                    full_config_item_key = '.'.join(parent_keys + [config_item_key])
+                    full_config_item_key = '.'.join(parent_keys + [str(config_item_key)])
                     value = self._get_datastore_value_for_expression(key=full_config_item_key,
                         value=config_item_value,
                         config_schema_item=schema_item)
@@ -188,8 +208,9 @@
             exc_class = type(e)
             original_msg = str(e)
             msg = ('Failed to render dynamic configuration value for key "%s" with value '
-                   '"%s" for pack "%s" config: %s ' % (key, value, self.pack_name, original_msg))
-            raise exc_class(msg)
+                   '"%s" for pack "%s" config: %s %s ' % (key, value, self.pack_name,
+                                                          exc_class, original_msg))
+            raise RuntimeError(msg)
 
         if value:
             # Deserialize the value

{"golden_diff": "diff --git a/st2common/st2common/util/config_loader.py b/st2common/st2common/util/config_loader.py\n--- a/st2common/st2common/util/config_loader.py\n+++ b/st2common/st2common/util/config_loader.py\n@@ -110,22 +110,42 @@\n         \"\"\"\n         parent_keys = parent_keys or []\n \n-        for config_item_key, config_item_value in six.iteritems(config):\n-            schema_item = schema.get(config_item_key, {})\n+        config_is_dict = isinstance(config, dict)\n+        config_is_list = isinstance(config, list)\n+        iterator = six.iteritems(config) if config_is_dict else enumerate(config)\n+\n+        # config_item_key - if config_is_dict then this is the key in the dictionary\n+        #                   if config_is_list then this is the index of them item\n+        # config_item_value - the value of the key/index for the current item\n+        for config_item_key, config_item_value in iterator:\n+            if config_is_dict:\n+                # different schema for each key/value pair\n+                schema_item = schema.get(config_item_key, {})\n+            if config_is_list:\n+                # same schema is shared between every item in the list\n+                schema_item = schema\n+\n             is_dictionary = isinstance(config_item_value, dict)\n+            is_list = isinstance(config_item_value, list)\n \n             # Inspect nested object properties\n             if is_dictionary:\n-                parent_keys += [config_item_key]\n+                parent_keys += [str(config_item_key)]\n                 self._assign_dynamic_config_values(schema=schema_item.get('properties', {}),\n                                                    config=config[config_item_key],\n                                                    parent_keys=parent_keys)\n+            # Inspect nested list items\n+            elif is_list:\n+                parent_keys += [str(config_item_key)]\n+                self._assign_dynamic_config_values(schema=schema_item.get('items', {}),\n+                                                   config=config[config_item_key],\n+                                                   parent_keys=parent_keys)\n             else:\n                 is_jinja_expression = jinja_utils.is_jinja_expression(value=config_item_value)\n \n                 if is_jinja_expression:\n                     # Resolve / render the Jinja template expression\n-                    full_config_item_key = '.'.join(parent_keys + [config_item_key])\n+                    full_config_item_key = '.'.join(parent_keys + [str(config_item_key)])\n                     value = self._get_datastore_value_for_expression(key=full_config_item_key,\n                         value=config_item_value,\n                         config_schema_item=schema_item)\n@@ -188,8 +208,9 @@\n             exc_class = type(e)\n             original_msg = str(e)\n             msg = ('Failed to render dynamic configuration value for key \"%s\" with value '\n-                   '\"%s\" for pack \"%s\" config: %s ' % (key, value, self.pack_name, original_msg))\n-            raise exc_class(msg)\n+                   '\"%s\" for pack \"%s\" config: %s %s ' % (key, value, self.pack_name,\n+                                                          exc_class, original_msg))\n+            raise RuntimeError(msg)\n \n         if value:\n             # Deserialize the value\n", "issue": "Dynamic Configuration values don't work with nested objects\nI use the following cmd to set a key in the datastore.\r\n`$ st2 key set -e smtp_account.me.password \"I'm Encrypted\"`\r\n\r\nand here are my email pack config /opt/stackstorm/configs/email.yaml\r\n\r\n```yaml\r\n---\r\nsmtp_accounts:\r\n  - name: name\r\n    password: \"{{ st2kv.system.smtp_account.me.password }}\"\r\n```\r\n(followed by the rest of the account configuration).\r\n\r\nThis is based on the email pack's `config.schema.yaml`, which contains this:\r\n\r\n```yaml\r\nimap_accounts:\r\n  description: \"IMAP accounts\"\r\n  type: \"array\"\r\n  required: true\r\n  items:\r\n    type: \"object\"\r\n    required: true\r\n    properties:\r\n      name:\r\n        description: \"Name of the account\"\r\n        type: \"string\"\r\n        secret: false\r\n        required: true\r\n      server:\r\n        description: \"Email server name - e.g. imap.gmail.com\"\r\n        type: \"string\"\r\n        secret: false\r\n        required: true\r\n      username:\r\n        description: \"Mailbox username\"\r\n        type: \"string\"\r\n        required: true\r\n      password:\r\n        description: \"Mailbox password.\"\r\n        type: \"string\"\r\n        secret: true\r\n        required: true\r\n```\r\n\r\nRunning the email action fails:\r\n```\r\nTraceback (most recent call last):\r\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py\", line 278, in <module>\r\n    obj.run()\r\n  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py\", line 171, in run\r\n    output = action.run(**self._parameters)\r\n  File \"/opt/stackstorm/packs/email/actions/send_email.py\", line 49, in run\r\n    s.login(account_data['username'], account_data['password'])\r\n  File \"/usr/lib64/python2.7/smtplib.py\", line 621, in login\r\n    raise SMTPAuthenticationError(code, resp)\r\nsmtplib.SMTPAuthenticationError: (460, 'ERR.LOGIN.PASSERR')\r\n```\r\n\r\nI added some log to see the password value in File \"/opt/stackstorm/packs/email/actions/send_email.py\", line 49.\r\nIt seems that we get\r\n`{{ st2kv.system.smtp_account.me.password }}` as the password.\r\n\r\nFYI\r\nIf I run the send email action and use `{{ st2kv.system.smtp_account.me.password }}` as the message\r\nI can see the message's value is correct.\n", "before_files": [{"content": "# Licensed to the StackStorm, Inc ('StackStorm') under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom __future__ import absolute_import\nimport copy\n\nimport six\n\nfrom oslo_config import cfg\n\nfrom st2common import log as logging\nfrom st2common.models.db.pack import ConfigDB\nfrom st2common.persistence.pack import ConfigSchema\nfrom st2common.persistence.pack import Config\nfrom st2common.content import utils as content_utils\nfrom st2common.util import jinja as jinja_utils\nfrom st2common.util.templating import render_template_with_system_and_user_context\nfrom st2common.util.config_parser import ContentPackConfigParser\nfrom st2common.exceptions.db import StackStormDBObjectNotFoundError\n\n__all__ = [\n    'ContentPackConfigLoader'\n]\n\nLOG = logging.getLogger(__name__)\n\n\nclass ContentPackConfigLoader(object):\n    \"\"\"\n    Class which loads and resolves all the config values and returns a dictionary of resolved values\n    which can be passed to the resource.\n\n    It loads and resolves values in the following order:\n\n    1. Static values from <pack path>/config.yaml file\n    2. Dynamic and or static values from /opt/stackstorm/configs/<pack name>.yaml file.\n\n    Values are merged from left to right which means values from \"<pack name>.yaml\" file have\n    precedence and override values from pack local config file.\n    \"\"\"\n\n    def __init__(self, pack_name, user=None):\n        self.pack_name = pack_name\n        self.user = user or cfg.CONF.system_user.user\n\n        self.pack_path = content_utils.get_pack_base_path(pack_name=pack_name)\n        self._config_parser = ContentPackConfigParser(pack_name=pack_name)\n\n    def get_config(self):\n        result = {}\n\n        # Retrieve corresponding ConfigDB and ConfigSchemaDB object\n        # Note: ConfigSchemaDB is optional right now. If it doesn't exist, we assume every value\n        # is of a type string\n        try:\n            config_db = Config.get_by_pack(value=self.pack_name)\n        except StackStormDBObjectNotFoundError:\n            # Corresponding pack config doesn't exist. We set config_db to an empty config so\n            # that the default values from config schema are still correctly applied even if\n            # pack doesn't contain a config.\n            config_db = ConfigDB(pack=self.pack_name, values={})\n\n        try:\n            config_schema_db = ConfigSchema.get_by_pack(value=self.pack_name)\n        except StackStormDBObjectNotFoundError:\n            config_schema_db = None\n\n        # 2. Retrieve values from \"global\" pack config file (if available) and resolve them if\n        # necessary\n        config = self._get_values_for_config(config_schema_db=config_schema_db,\n                                             config_db=config_db)\n        result.update(config)\n\n        return result\n\n    def _get_values_for_config(self, config_schema_db, config_db):\n        schema_values = getattr(config_schema_db, 'attributes', {})\n        config_values = getattr(config_db, 'values', {})\n\n        config = copy.deepcopy(config_values)\n\n        # Assign dynamic config values based on the values in the datastore\n        config = self._assign_dynamic_config_values(schema=schema_values, config=config)\n\n        # If config_schema is available we do a second pass and set default values for required\n        # items which values are not provided / available in the config itself\n        config = self._assign_default_values(schema=schema_values, config=config)\n        return config\n\n    def _assign_dynamic_config_values(self, schema, config, parent_keys=None):\n        \"\"\"\n        Assign dynamic config value for a particular config item if the ite utilizes a Jinja\n        expression for dynamic config values.\n\n        Note: This method mutates config argument in place.\n\n        :rtype: ``dict``\n        \"\"\"\n        parent_keys = parent_keys or []\n\n        for config_item_key, config_item_value in six.iteritems(config):\n            schema_item = schema.get(config_item_key, {})\n            is_dictionary = isinstance(config_item_value, dict)\n\n            # Inspect nested object properties\n            if is_dictionary:\n                parent_keys += [config_item_key]\n                self._assign_dynamic_config_values(schema=schema_item.get('properties', {}),\n                                                   config=config[config_item_key],\n                                                   parent_keys=parent_keys)\n            else:\n                is_jinja_expression = jinja_utils.is_jinja_expression(value=config_item_value)\n\n                if is_jinja_expression:\n                    # Resolve / render the Jinja template expression\n                    full_config_item_key = '.'.join(parent_keys + [config_item_key])\n                    value = self._get_datastore_value_for_expression(key=full_config_item_key,\n                        value=config_item_value,\n                        config_schema_item=schema_item)\n\n                    config[config_item_key] = value\n                else:\n                    # Static value, no resolution needed\n                    config[config_item_key] = config_item_value\n\n        return config\n\n    def _assign_default_values(self, schema, config):\n        \"\"\"\n        Assign default values for particular config if default values are provided in the config\n        schema and a value is not specified in the config.\n\n        Note: This method mutates config argument in place.\n\n        :rtype: ``dict``\n        \"\"\"\n        for schema_item_key, schema_item in six.iteritems(schema):\n            has_default_value = 'default' in schema_item\n            has_config_value = schema_item_key in config\n\n            default_value = schema_item.get('default', None)\n            is_object = schema_item.get('type', None) == 'object'\n            has_properties = schema_item.get('properties', None)\n\n            if has_default_value and not has_config_value:\n                # Config value is not provided, but default value is, use a default value\n                config[schema_item_key] = default_value\n\n            # Inspect nested object properties\n            if is_object and has_properties:\n                if not config.get(schema_item_key, None):\n                    config[schema_item_key] = {}\n\n                self._assign_default_values(schema=schema_item['properties'],\n                                            config=config[schema_item_key])\n\n        return config\n\n    def _get_datastore_value_for_expression(self, key, value, config_schema_item=None):\n        \"\"\"\n        Retrieve datastore value by first resolving the datastore expression and then retrieving\n        the value from the datastore.\n\n        :param key: Full path to the config item key (e.g. \"token\" / \"auth.settings.token\", etc.)\n        \"\"\"\n        from st2common.services.config import deserialize_key_value\n\n        config_schema_item = config_schema_item or {}\n        secret = config_schema_item.get('secret', False)\n\n        try:\n            value = render_template_with_system_and_user_context(value=value,\n                                                                 user=self.user)\n        except Exception as e:\n            # Throw a more user-friendly exception on failed render\n            exc_class = type(e)\n            original_msg = str(e)\n            msg = ('Failed to render dynamic configuration value for key \"%s\" with value '\n                   '\"%s\" for pack \"%s\" config: %s ' % (key, value, self.pack_name, original_msg))\n            raise exc_class(msg)\n\n        if value:\n            # Deserialize the value\n            value = deserialize_key_value(value=value, secret=secret)\n        else:\n            value = None\n\n        return value\n\n\ndef get_config(pack, user):\n    \"\"\"Returns config for given pack and user.\n    \"\"\"\n    LOG.debug('Attempting to get config')\n    if pack and user:\n        LOG.debug('Pack and user found. Loading config.')\n        config_loader = ContentPackConfigLoader(\n            pack_name=pack,\n            user=user\n        )\n\n        config = config_loader.get_config()\n\n    else:\n        config = {}\n\n    LOG.debug('Config: %s', config)\n\n    return config\n", "path": "st2common/st2common/util/config_loader.py"}], "after_files": [{"content": "# Licensed to the StackStorm, Inc ('StackStorm') under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom __future__ import absolute_import\nimport copy\n\nimport six\n\nfrom oslo_config import cfg\n\nfrom st2common import log as logging\nfrom st2common.models.db.pack import ConfigDB\nfrom st2common.persistence.pack import ConfigSchema\nfrom st2common.persistence.pack import Config\nfrom st2common.content import utils as content_utils\nfrom st2common.util import jinja as jinja_utils\nfrom st2common.util.templating import render_template_with_system_and_user_context\nfrom st2common.util.config_parser import ContentPackConfigParser\nfrom st2common.exceptions.db import StackStormDBObjectNotFoundError\n\n__all__ = [\n    'ContentPackConfigLoader'\n]\n\nLOG = logging.getLogger(__name__)\n\n\nclass ContentPackConfigLoader(object):\n    \"\"\"\n    Class which loads and resolves all the config values and returns a dictionary of resolved values\n    which can be passed to the resource.\n\n    It loads and resolves values in the following order:\n\n    1. Static values from <pack path>/config.yaml file\n    2. Dynamic and or static values from /opt/stackstorm/configs/<pack name>.yaml file.\n\n    Values are merged from left to right which means values from \"<pack name>.yaml\" file have\n    precedence and override values from pack local config file.\n    \"\"\"\n\n    def __init__(self, pack_name, user=None):\n        self.pack_name = pack_name\n        self.user = user or cfg.CONF.system_user.user\n\n        self.pack_path = content_utils.get_pack_base_path(pack_name=pack_name)\n        self._config_parser = ContentPackConfigParser(pack_name=pack_name)\n\n    def get_config(self):\n        result = {}\n\n        # Retrieve corresponding ConfigDB and ConfigSchemaDB object\n        # Note: ConfigSchemaDB is optional right now. If it doesn't exist, we assume every value\n        # is of a type string\n        try:\n            config_db = Config.get_by_pack(value=self.pack_name)\n        except StackStormDBObjectNotFoundError:\n            # Corresponding pack config doesn't exist. We set config_db to an empty config so\n            # that the default values from config schema are still correctly applied even if\n            # pack doesn't contain a config.\n            config_db = ConfigDB(pack=self.pack_name, values={})\n\n        try:\n            config_schema_db = ConfigSchema.get_by_pack(value=self.pack_name)\n        except StackStormDBObjectNotFoundError:\n            config_schema_db = None\n\n        # 2. Retrieve values from \"global\" pack config file (if available) and resolve them if\n        # necessary\n        config = self._get_values_for_config(config_schema_db=config_schema_db,\n                                             config_db=config_db)\n        result.update(config)\n\n        return result\n\n    def _get_values_for_config(self, config_schema_db, config_db):\n        schema_values = getattr(config_schema_db, 'attributes', {})\n        config_values = getattr(config_db, 'values', {})\n\n        config = copy.deepcopy(config_values)\n\n        # Assign dynamic config values based on the values in the datastore\n        config = self._assign_dynamic_config_values(schema=schema_values, config=config)\n\n        # If config_schema is available we do a second pass and set default values for required\n        # items which values are not provided / available in the config itself\n        config = self._assign_default_values(schema=schema_values, config=config)\n        return config\n\n    def _assign_dynamic_config_values(self, schema, config, parent_keys=None):\n        \"\"\"\n        Assign dynamic config value for a particular config item if the ite utilizes a Jinja\n        expression for dynamic config values.\n\n        Note: This method mutates config argument in place.\n\n        :rtype: ``dict``\n        \"\"\"\n        parent_keys = parent_keys or []\n\n        config_is_dict = isinstance(config, dict)\n        config_is_list = isinstance(config, list)\n        iterator = six.iteritems(config) if config_is_dict else enumerate(config)\n\n        # config_item_key - if config_is_dict then this is the key in the dictionary\n        #                   if config_is_list then this is the index of them item\n        # config_item_value - the value of the key/index for the current item\n        for config_item_key, config_item_value in iterator:\n            if config_is_dict:\n                # different schema for each key/value pair\n                schema_item = schema.get(config_item_key, {})\n            if config_is_list:\n                # same schema is shared between every item in the list\n                schema_item = schema\n\n            is_dictionary = isinstance(config_item_value, dict)\n            is_list = isinstance(config_item_value, list)\n\n            # Inspect nested object properties\n            if is_dictionary:\n                parent_keys += [str(config_item_key)]\n                self._assign_dynamic_config_values(schema=schema_item.get('properties', {}),\n                                                   config=config[config_item_key],\n                                                   parent_keys=parent_keys)\n            # Inspect nested list items\n            elif is_list:\n                parent_keys += [str(config_item_key)]\n                self._assign_dynamic_config_values(schema=schema_item.get('items', {}),\n                                                   config=config[config_item_key],\n                                                   parent_keys=parent_keys)\n            else:\n                is_jinja_expression = jinja_utils.is_jinja_expression(value=config_item_value)\n\n                if is_jinja_expression:\n                    # Resolve / render the Jinja template expression\n                    full_config_item_key = '.'.join(parent_keys + [str(config_item_key)])\n                    value = self._get_datastore_value_for_expression(key=full_config_item_key,\n                        value=config_item_value,\n                        config_schema_item=schema_item)\n\n                    config[config_item_key] = value\n                else:\n                    # Static value, no resolution needed\n                    config[config_item_key] = config_item_value\n\n        return config\n\n    def _assign_default_values(self, schema, config):\n        \"\"\"\n        Assign default values for particular config if default values are provided in the config\n        schema and a value is not specified in the config.\n\n        Note: This method mutates config argument in place.\n\n        :rtype: ``dict``\n        \"\"\"\n        for schema_item_key, schema_item in six.iteritems(schema):\n            has_default_value = 'default' in schema_item\n            has_config_value = schema_item_key in config\n\n            default_value = schema_item.get('default', None)\n            is_object = schema_item.get('type', None) == 'object'\n            has_properties = schema_item.get('properties', None)\n\n            if has_default_value and not has_config_value:\n                # Config value is not provided, but default value is, use a default value\n                config[schema_item_key] = default_value\n\n            # Inspect nested object properties\n            if is_object and has_properties:\n                if not config.get(schema_item_key, None):\n                    config[schema_item_key] = {}\n\n                self._assign_default_values(schema=schema_item['properties'],\n                                            config=config[schema_item_key])\n\n        return config\n\n    def _get_datastore_value_for_expression(self, key, value, config_schema_item=None):\n        \"\"\"\n        Retrieve datastore value by first resolving the datastore expression and then retrieving\n        the value from the datastore.\n\n        :param key: Full path to the config item key (e.g. \"token\" / \"auth.settings.token\", etc.)\n        \"\"\"\n        from st2common.services.config import deserialize_key_value\n\n        config_schema_item = config_schema_item or {}\n        secret = config_schema_item.get('secret', False)\n\n        try:\n            value = render_template_with_system_and_user_context(value=value,\n                                                                 user=self.user)\n        except Exception as e:\n            # Throw a more user-friendly exception on failed render\n            exc_class = type(e)\n            original_msg = str(e)\n            msg = ('Failed to render dynamic configuration value for key \"%s\" with value '\n                   '\"%s\" for pack \"%s\" config: %s %s ' % (key, value, self.pack_name,\n                                                          exc_class, original_msg))\n            raise RuntimeError(msg)\n\n        if value:\n            # Deserialize the value\n            value = deserialize_key_value(value=value, secret=secret)\n        else:\n            value = None\n\n        return value\n\n\ndef get_config(pack, user):\n    \"\"\"Returns config for given pack and user.\n    \"\"\"\n    LOG.debug('Attempting to get config')\n    if pack and user:\n        LOG.debug('Pack and user found. Loading config.')\n        config_loader = ContentPackConfigLoader(\n            pack_name=pack,\n            user=user\n        )\n\n        config = config_loader.get_config()\n\n    else:\n        config = {}\n\n    LOG.debug('Config: %s', config)\n\n    return config\n", "path": "st2common/st2common/util/config_loader.py"}]}