We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[tune] Feature request: tune.sample_from does not support callable objects. 
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 16.04
- **Ray installed from (source or binary)**: binary
- **Ray version**: 0.7.2
- **Python version**: 3.2
- **Exact command to reproduce**: See below

### Describe the problem

The `tune` sample_from interface is strictly limited to function objects, such as lambdas. This serves most use cases, but there are a number of instances where it's very useful to define a callable object to yield samples. (See trivial example below.) At the moment, providing a callable object returns errors from within tune variant generation, as the non-function-based `sample_from` entries are processed in grid entries. This can be resolved by changeing the sample/grid check from a direct check for `FunctionType` (Source location: https://github.com/ray-project/ray/blob/fadfa5f30bb654a74c781eaf8396a35af3ab7760/python/ray/tune/suggest/variant_generator.py#L116) to the builtin function `callable`.

I'm not entirely clear if this is an intentional limitation, and changing this logic will likely require expansion of tune's tests and documentation to cover the new behavior. I would be happy to open a PR for this if a maintainer gives the feature a 👍.

### Source code / logs

```python

import random

import ray.tune as tune
from ray.tune.suggest.variant_generator import generate_variants

class Normal:
    def __call__(self, _config):
        return random.normalvariate(mu=0, sigma=1)


grid_config = {"grid": tune.grid_search(list(range(2)))}
sample_config = {"normal": tune.sample_from(Normal())}


print(grid_config)
print(list(generate_variants(grid_config)))

print(sample_config)
print(list(generate_variants(sample_config)))

```

Results:

```
{'grid': {'grid_search': [0, 1]}}
[('grid=0', {'grid': 0}), ('grid=1', {'grid': 1})]
{'normal': tune.sample_from(<__main__.Normal object at 0x7f08ed1d0f50>)}
Traceback (most recent call last):
  File "sample_error.py", line 19, in <module>
    print(list(generate_variants(sample_config)))
  File "/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py", line 43, in generate_variants
    for resolved_vars, spec in _generate_variants(unresolved_spec):
  File "/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py", line 123, in _generate_variants
    for resolved_spec in grid_search:
  File "/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py", line 193, in _grid_search_generator
    while value_indices[-1] < len(grid_vars[-1][1]):
TypeError: object of type 'Normal' has no len()
```


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `python/ray/tune/suggest/variant_generator.py`
Content:
```
1 import copy
2 import logging
3 import numpy
4 import random
5 import types
6 
7 from ray.tune import TuneError
8 from ray.tune.sample import sample_from
9 
10 logger = logging.getLogger(__name__)
11 
12 
13 def generate_variants(unresolved_spec):
14     """Generates variants from a spec (dict) with unresolved values.
15 
16     There are two types of unresolved values:
17 
18         Grid search: These define a grid search over values. For example, the
19         following grid search values in a spec will produce six distinct
20         variants in combination:
21 
22             "activation": grid_search(["relu", "tanh"])
23             "learning_rate": grid_search([1e-3, 1e-4, 1e-5])
24 
25         Lambda functions: These are evaluated to produce a concrete value, and
26         can express dependencies or conditional distributions between values.
27         They can also be used to express random search (e.g., by calling
28         into the `random` or `np` module).
29 
30             "cpu": lambda spec: spec.config.num_workers
31             "batch_size": lambda spec: random.uniform(1, 1000)
32 
33     Finally, to support defining specs in plain JSON / YAML, grid search
34     and lambda functions can also be defined alternatively as follows:
35 
36         "activation": {"grid_search": ["relu", "tanh"]}
37         "cpu": {"eval": "spec.config.num_workers"}
38 
39     Use `format_vars` to format the returned dict of hyperparameters.
40 
41     Yields:
42         (Dict of resolved variables, Spec object)
43     """
44     for resolved_vars, spec in _generate_variants(unresolved_spec):
45         assert not _unresolved_values(spec)
46         yield resolved_vars, spec
47 
48 
49 def grid_search(values):
50     """Convenience method for specifying grid search over a value.
51 
52     Arguments:
53         values: An iterable whose parameters will be gridded.
54     """
55 
56     return {"grid_search": values}
57 
58 
59 _STANDARD_IMPORTS = {
60     "random": random,
61     "np": numpy,
62 }
63 
64 _MAX_RESOLUTION_PASSES = 20
65 
66 
67 def resolve_nested_dict(nested_dict):
68     """Flattens a nested dict by joining keys into tuple of paths.
69 
70     Can then be passed into `format_vars`.
71     """
72     res = {}
73     for k, v in nested_dict.items():
74         if isinstance(v, dict):
75             for k_, v_ in resolve_nested_dict(v).items():
76                 res[(k, ) + k_] = v_
77         else:
78             res[(k, )] = v
79     return res
80 
81 
82 def format_vars(resolved_vars):
83     """Formats the resolved variable dict into a single string."""
84     out = []
85     for path, value in sorted(resolved_vars.items()):
86         if path[0] in ["run", "env", "resources_per_trial"]:
87             continue  # TrialRunner already has these in the experiment_tag
88         pieces = []
89         last_string = True
90         for k in path[::-1]:
91             if isinstance(k, int):
92                 pieces.append(str(k))
93             elif last_string:
94                 last_string = False
95                 pieces.append(k)
96         pieces.reverse()
97         out.append(_clean_value("_".join(pieces)) + "=" + _clean_value(value))
98     return ",".join(out)
99 
100 
101 def flatten_resolved_vars(resolved_vars):
102     """Formats the resolved variable dict into a mapping of (str -> value)."""
103     flattened_resolved_vars_dict = {}
104     for pieces, value in resolved_vars.items():
105         if pieces[0] == "config":
106             pieces = pieces[1:]
107         pieces = [str(piece) for piece in pieces]
108         flattened_resolved_vars_dict["/".join(pieces)] = value
109     return flattened_resolved_vars_dict
110 
111 
112 def _clean_value(value):
113     if isinstance(value, float):
114         return "{:.5}".format(value)
115     else:
116         return str(value).replace("/", "_")
117 
118 
119 def _generate_variants(spec):
120     spec = copy.deepcopy(spec)
121     unresolved = _unresolved_values(spec)
122     if not unresolved:
123         yield {}, spec
124         return
125 
126     grid_vars = []
127     lambda_vars = []
128     for path, value in unresolved.items():
129         if isinstance(value, types.FunctionType):
130             lambda_vars.append((path, value))
131         else:
132             grid_vars.append((path, value))
133     grid_vars.sort()
134 
135     grid_search = _grid_search_generator(spec, grid_vars)
136     for resolved_spec in grid_search:
137         resolved_vars = _resolve_lambda_vars(resolved_spec, lambda_vars)
138         for resolved, spec in _generate_variants(resolved_spec):
139             for path, value in grid_vars:
140                 resolved_vars[path] = _get_value(spec, path)
141             for k, v in resolved.items():
142                 if (k in resolved_vars and v != resolved_vars[k]
143                         and _is_resolved(resolved_vars[k])):
144                     raise ValueError(
145                         "The variable `{}` could not be unambiguously "
146                         "resolved to a single value. Consider simplifying "
147                         "your configuration.".format(k))
148                 resolved_vars[k] = v
149             yield resolved_vars, spec
150 
151 
152 def _assign_value(spec, path, value):
153     for k in path[:-1]:
154         spec = spec[k]
155     spec[path[-1]] = value
156 
157 
158 def _get_value(spec, path):
159     for k in path:
160         spec = spec[k]
161     return spec
162 
163 
164 def _resolve_lambda_vars(spec, lambda_vars):
165     resolved = {}
166     error = True
167     num_passes = 0
168     while error and num_passes < _MAX_RESOLUTION_PASSES:
169         num_passes += 1
170         error = False
171         for path, fn in lambda_vars:
172             try:
173                 value = fn(_UnresolvedAccessGuard(spec))
174             except RecursiveDependencyError as e:
175                 error = e
176             except Exception:
177                 raise ValueError(
178                     "Failed to evaluate expression: {}: {}".format(path, fn))
179             else:
180                 _assign_value(spec, path, value)
181                 resolved[path] = value
182     if error:
183         raise error
184     return resolved
185 
186 
187 def _grid_search_generator(unresolved_spec, grid_vars):
188     value_indices = [0] * len(grid_vars)
189 
190     def increment(i):
191         value_indices[i] += 1
192         if value_indices[i] >= len(grid_vars[i][1]):
193             value_indices[i] = 0
194             if i + 1 < len(value_indices):
195                 return increment(i + 1)
196             else:
197                 return True
198         return False
199 
200     if not grid_vars:
201         yield unresolved_spec
202         return
203 
204     while value_indices[-1] < len(grid_vars[-1][1]):
205         spec = copy.deepcopy(unresolved_spec)
206         for i, (path, values) in enumerate(grid_vars):
207             _assign_value(spec, path, values[value_indices[i]])
208         yield spec
209         if grid_vars:
210             done = increment(0)
211             if done:
212                 break
213 
214 
215 def _is_resolved(v):
216     resolved, _ = _try_resolve(v)
217     return resolved
218 
219 
220 def _try_resolve(v):
221     if isinstance(v, sample_from):
222         # Function to sample from
223         return False, v.func
224     elif isinstance(v, dict) and len(v) == 1 and "eval" in v:
225         # Lambda function in eval syntax
226         return False, lambda spec: eval(
227             v["eval"], _STANDARD_IMPORTS, {"spec": spec})
228     elif isinstance(v, dict) and len(v) == 1 and "grid_search" in v:
229         # Grid search values
230         grid_values = v["grid_search"]
231         if not isinstance(grid_values, list):
232             raise TuneError(
233                 "Grid search expected list of values, got: {}".format(
234                     grid_values))
235         return False, grid_values
236     return True, v
237 
238 
239 def _unresolved_values(spec):
240     found = {}
241     for k, v in spec.items():
242         resolved, v = _try_resolve(v)
243         if not resolved:
244             found[(k, )] = v
245         elif isinstance(v, dict):
246             # Recurse into a dict
247             for (path, value) in _unresolved_values(v).items():
248                 found[(k, ) + path] = value
249         elif isinstance(v, list):
250             # Recurse into a list
251             for i, elem in enumerate(v):
252                 for (path, value) in _unresolved_values({i: elem}).items():
253                     found[(k, ) + path] = value
254     return found
255 
256 
257 class _UnresolvedAccessGuard(dict):
258     def __init__(self, *args, **kwds):
259         super(_UnresolvedAccessGuard, self).__init__(*args, **kwds)
260         self.__dict__ = self
261 
262     def __getattribute__(self, item):
263         value = dict.__getattribute__(self, item)
264         if not _is_resolved(value):
265             raise RecursiveDependencyError(
266                 "`{}` recursively depends on {}".format(item, value))
267         elif isinstance(value, dict):
268             return _UnresolvedAccessGuard(value)
269         else:
270             return value
271 
272 
273 class RecursiveDependencyError(Exception):
274     def __init__(self, msg):
275         Exception.__init__(self, msg)
276
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/python/ray/tune/suggest/variant_generator.py b/python/ray/tune/suggest/variant_generator.py
--- a/python/ray/tune/suggest/variant_generator.py
+++ b/python/ray/tune/suggest/variant_generator.py
@@ -2,7 +2,6 @@
 import logging
 import numpy
 import random
-import types
 
 from ray.tune import TuneError
 from ray.tune.sample import sample_from
@@ -126,7 +125,7 @@
     grid_vars = []
     lambda_vars = []
     for path, value in unresolved.items():
-        if isinstance(value, types.FunctionType):
+        if callable(value):
             lambda_vars.append((path, value))
         else:
             grid_vars.append((path, value))

{"golden_diff": "diff --git a/python/ray/tune/suggest/variant_generator.py b/python/ray/tune/suggest/variant_generator.py\n--- a/python/ray/tune/suggest/variant_generator.py\n+++ b/python/ray/tune/suggest/variant_generator.py\n@@ -2,7 +2,6 @@\n import logging\n import numpy\n import random\n-import types\n \n from ray.tune import TuneError\n from ray.tune.sample import sample_from\n@@ -126,7 +125,7 @@\n     grid_vars = []\n     lambda_vars = []\n     for path, value in unresolved.items():\n-        if isinstance(value, types.FunctionType):\n+        if callable(value):\n             lambda_vars.append((path, value))\n         else:\n             grid_vars.append((path, value))\n", "issue": "[tune] Feature request: tune.sample_from does not support callable objects. \n### System information\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 16.04\r\n- **Ray installed from (source or binary)**: binary\r\n- **Ray version**: 0.7.2\r\n- **Python version**: 3.2\r\n- **Exact command to reproduce**: See below\r\n\r\n### Describe the problem\r\n\r\nThe `tune` sample_from interface is strictly limited to function objects, such as lambdas. This serves most use cases, but there are a number of instances where it's very useful to define a callable object to yield samples. (See trivial example below.) At the moment, providing a callable object returns errors from within tune variant generation, as the non-function-based `sample_from` entries are processed in grid entries. This can be resolved by changeing the sample/grid check from a direct check for `FunctionType` (Source location: https://github.com/ray-project/ray/blob/fadfa5f30bb654a74c781eaf8396a35af3ab7760/python/ray/tune/suggest/variant_generator.py#L116) to the builtin function `callable`.\r\n\r\nI'm not entirely clear if this is an intentional limitation, and changing this logic will likely require expansion of tune's tests and documentation to cover the new behavior. I would be happy to open a PR for this if a maintainer gives the feature a \ud83d\udc4d.\r\n\r\n### Source code / logs\r\n\r\n```python\r\n\r\nimport random\r\n\r\nimport ray.tune as tune\r\nfrom ray.tune.suggest.variant_generator import generate_variants\r\n\r\nclass Normal:\r\n    def __call__(self, _config):\r\n        return random.normalvariate(mu=0, sigma=1)\r\n\r\n\r\ngrid_config = {\"grid\": tune.grid_search(list(range(2)))}\r\nsample_config = {\"normal\": tune.sample_from(Normal())}\r\n\r\n\r\nprint(grid_config)\r\nprint(list(generate_variants(grid_config)))\r\n\r\nprint(sample_config)\r\nprint(list(generate_variants(sample_config)))\r\n\r\n```\r\n\r\nResults:\r\n\r\n```\r\n{'grid': {'grid_search': [0, 1]}}\r\n[('grid=0', {'grid': 0}), ('grid=1', {'grid': 1})]\r\n{'normal': tune.sample_from(<__main__.Normal object at 0x7f08ed1d0f50>)}\r\nTraceback (most recent call last):\r\n  File \"sample_error.py\", line 19, in <module>\r\n    print(list(generate_variants(sample_config)))\r\n  File \"/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py\", line 43, in generate_variants\r\n    for resolved_vars, spec in _generate_variants(unresolved_spec):\r\n  File \"/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py\", line 123, in _generate_variants\r\n    for resolved_spec in grid_search:\r\n  File \"/work/home/lexaf/workspace/alphabeta/.conda/lib/python3.7/site-packages/ray/tune/suggest/variant_generator.py\", line 193, in _grid_search_generator\r\n    while value_indices[-1] < len(grid_vars[-1][1]):\r\nTypeError: object of type 'Normal' has no len()\r\n```\r\n\n", "before_files": [{"content": "import copy\nimport logging\nimport numpy\nimport random\nimport types\n\nfrom ray.tune import TuneError\nfrom ray.tune.sample import sample_from\n\nlogger = logging.getLogger(__name__)\n\n\ndef generate_variants(unresolved_spec):\n    \"\"\"Generates variants from a spec (dict) with unresolved values.\n\n    There are two types of unresolved values:\n\n        Grid search: These define a grid search over values. For example, the\n        following grid search values in a spec will produce six distinct\n        variants in combination:\n\n            \"activation\": grid_search([\"relu\", \"tanh\"])\n            \"learning_rate\": grid_search([1e-3, 1e-4, 1e-5])\n\n        Lambda functions: These are evaluated to produce a concrete value, and\n        can express dependencies or conditional distributions between values.\n        They can also be used to express random search (e.g., by calling\n        into the `random` or `np` module).\n\n            \"cpu\": lambda spec: spec.config.num_workers\n            \"batch_size\": lambda spec: random.uniform(1, 1000)\n\n    Finally, to support defining specs in plain JSON / YAML, grid search\n    and lambda functions can also be defined alternatively as follows:\n\n        \"activation\": {\"grid_search\": [\"relu\", \"tanh\"]}\n        \"cpu\": {\"eval\": \"spec.config.num_workers\"}\n\n    Use `format_vars` to format the returned dict of hyperparameters.\n\n    Yields:\n        (Dict of resolved variables, Spec object)\n    \"\"\"\n    for resolved_vars, spec in _generate_variants(unresolved_spec):\n        assert not _unresolved_values(spec)\n        yield resolved_vars, spec\n\n\ndef grid_search(values):\n    \"\"\"Convenience method for specifying grid search over a value.\n\n    Arguments:\n        values: An iterable whose parameters will be gridded.\n    \"\"\"\n\n    return {\"grid_search\": values}\n\n\n_STANDARD_IMPORTS = {\n    \"random\": random,\n    \"np\": numpy,\n}\n\n_MAX_RESOLUTION_PASSES = 20\n\n\ndef resolve_nested_dict(nested_dict):\n    \"\"\"Flattens a nested dict by joining keys into tuple of paths.\n\n    Can then be passed into `format_vars`.\n    \"\"\"\n    res = {}\n    for k, v in nested_dict.items():\n        if isinstance(v, dict):\n            for k_, v_ in resolve_nested_dict(v).items():\n                res[(k, ) + k_] = v_\n        else:\n            res[(k, )] = v\n    return res\n\n\ndef format_vars(resolved_vars):\n    \"\"\"Formats the resolved variable dict into a single string.\"\"\"\n    out = []\n    for path, value in sorted(resolved_vars.items()):\n        if path[0] in [\"run\", \"env\", \"resources_per_trial\"]:\n            continue  # TrialRunner already has these in the experiment_tag\n        pieces = []\n        last_string = True\n        for k in path[::-1]:\n            if isinstance(k, int):\n                pieces.append(str(k))\n            elif last_string:\n                last_string = False\n                pieces.append(k)\n        pieces.reverse()\n        out.append(_clean_value(\"_\".join(pieces)) + \"=\" + _clean_value(value))\n    return \",\".join(out)\n\n\ndef flatten_resolved_vars(resolved_vars):\n    \"\"\"Formats the resolved variable dict into a mapping of (str -> value).\"\"\"\n    flattened_resolved_vars_dict = {}\n    for pieces, value in resolved_vars.items():\n        if pieces[0] == \"config\":\n            pieces = pieces[1:]\n        pieces = [str(piece) for piece in pieces]\n        flattened_resolved_vars_dict[\"/\".join(pieces)] = value\n    return flattened_resolved_vars_dict\n\n\ndef _clean_value(value):\n    if isinstance(value, float):\n        return \"{:.5}\".format(value)\n    else:\n        return str(value).replace(\"/\", \"_\")\n\n\ndef _generate_variants(spec):\n    spec = copy.deepcopy(spec)\n    unresolved = _unresolved_values(spec)\n    if not unresolved:\n        yield {}, spec\n        return\n\n    grid_vars = []\n    lambda_vars = []\n    for path, value in unresolved.items():\n        if isinstance(value, types.FunctionType):\n            lambda_vars.append((path, value))\n        else:\n            grid_vars.append((path, value))\n    grid_vars.sort()\n\n    grid_search = _grid_search_generator(spec, grid_vars)\n    for resolved_spec in grid_search:\n        resolved_vars = _resolve_lambda_vars(resolved_spec, lambda_vars)\n        for resolved, spec in _generate_variants(resolved_spec):\n            for path, value in grid_vars:\n                resolved_vars[path] = _get_value(spec, path)\n            for k, v in resolved.items():\n                if (k in resolved_vars and v != resolved_vars[k]\n                        and _is_resolved(resolved_vars[k])):\n                    raise ValueError(\n                        \"The variable `{}` could not be unambiguously \"\n                        \"resolved to a single value. Consider simplifying \"\n                        \"your configuration.\".format(k))\n                resolved_vars[k] = v\n            yield resolved_vars, spec\n\n\ndef _assign_value(spec, path, value):\n    for k in path[:-1]:\n        spec = spec[k]\n    spec[path[-1]] = value\n\n\ndef _get_value(spec, path):\n    for k in path:\n        spec = spec[k]\n    return spec\n\n\ndef _resolve_lambda_vars(spec, lambda_vars):\n    resolved = {}\n    error = True\n    num_passes = 0\n    while error and num_passes < _MAX_RESOLUTION_PASSES:\n        num_passes += 1\n        error = False\n        for path, fn in lambda_vars:\n            try:\n                value = fn(_UnresolvedAccessGuard(spec))\n            except RecursiveDependencyError as e:\n                error = e\n            except Exception:\n                raise ValueError(\n                    \"Failed to evaluate expression: {}: {}\".format(path, fn))\n            else:\n                _assign_value(spec, path, value)\n                resolved[path] = value\n    if error:\n        raise error\n    return resolved\n\n\ndef _grid_search_generator(unresolved_spec, grid_vars):\n    value_indices = [0] * len(grid_vars)\n\n    def increment(i):\n        value_indices[i] += 1\n        if value_indices[i] >= len(grid_vars[i][1]):\n            value_indices[i] = 0\n            if i + 1 < len(value_indices):\n                return increment(i + 1)\n            else:\n                return True\n        return False\n\n    if not grid_vars:\n        yield unresolved_spec\n        return\n\n    while value_indices[-1] < len(grid_vars[-1][1]):\n        spec = copy.deepcopy(unresolved_spec)\n        for i, (path, values) in enumerate(grid_vars):\n            _assign_value(spec, path, values[value_indices[i]])\n        yield spec\n        if grid_vars:\n            done = increment(0)\n            if done:\n                break\n\n\ndef _is_resolved(v):\n    resolved, _ = _try_resolve(v)\n    return resolved\n\n\ndef _try_resolve(v):\n    if isinstance(v, sample_from):\n        # Function to sample from\n        return False, v.func\n    elif isinstance(v, dict) and len(v) == 1 and \"eval\" in v:\n        # Lambda function in eval syntax\n        return False, lambda spec: eval(\n            v[\"eval\"], _STANDARD_IMPORTS, {\"spec\": spec})\n    elif isinstance(v, dict) and len(v) == 1 and \"grid_search\" in v:\n        # Grid search values\n        grid_values = v[\"grid_search\"]\n        if not isinstance(grid_values, list):\n            raise TuneError(\n                \"Grid search expected list of values, got: {}\".format(\n                    grid_values))\n        return False, grid_values\n    return True, v\n\n\ndef _unresolved_values(spec):\n    found = {}\n    for k, v in spec.items():\n        resolved, v = _try_resolve(v)\n        if not resolved:\n            found[(k, )] = v\n        elif isinstance(v, dict):\n            # Recurse into a dict\n            for (path, value) in _unresolved_values(v).items():\n                found[(k, ) + path] = value\n        elif isinstance(v, list):\n            # Recurse into a list\n            for i, elem in enumerate(v):\n                for (path, value) in _unresolved_values({i: elem}).items():\n                    found[(k, ) + path] = value\n    return found\n\n\nclass _UnresolvedAccessGuard(dict):\n    def __init__(self, *args, **kwds):\n        super(_UnresolvedAccessGuard, self).__init__(*args, **kwds)\n        self.__dict__ = self\n\n    def __getattribute__(self, item):\n        value = dict.__getattribute__(self, item)\n        if not _is_resolved(value):\n            raise RecursiveDependencyError(\n                \"`{}` recursively depends on {}\".format(item, value))\n        elif isinstance(value, dict):\n            return _UnresolvedAccessGuard(value)\n        else:\n            return value\n\n\nclass RecursiveDependencyError(Exception):\n    def __init__(self, msg):\n        Exception.__init__(self, msg)\n", "path": "python/ray/tune/suggest/variant_generator.py"}], "after_files": [{"content": "import copy\nimport logging\nimport numpy\nimport random\n\nfrom ray.tune import TuneError\nfrom ray.tune.sample import sample_from\n\nlogger = logging.getLogger(__name__)\n\n\ndef generate_variants(unresolved_spec):\n    \"\"\"Generates variants from a spec (dict) with unresolved values.\n\n    There are two types of unresolved values:\n\n        Grid search: These define a grid search over values. For example, the\n        following grid search values in a spec will produce six distinct\n        variants in combination:\n\n            \"activation\": grid_search([\"relu\", \"tanh\"])\n            \"learning_rate\": grid_search([1e-3, 1e-4, 1e-5])\n\n        Lambda functions: These are evaluated to produce a concrete value, and\n        can express dependencies or conditional distributions between values.\n        They can also be used to express random search (e.g., by calling\n        into the `random` or `np` module).\n\n            \"cpu\": lambda spec: spec.config.num_workers\n            \"batch_size\": lambda spec: random.uniform(1, 1000)\n\n    Finally, to support defining specs in plain JSON / YAML, grid search\n    and lambda functions can also be defined alternatively as follows:\n\n        \"activation\": {\"grid_search\": [\"relu\", \"tanh\"]}\n        \"cpu\": {\"eval\": \"spec.config.num_workers\"}\n\n    Use `format_vars` to format the returned dict of hyperparameters.\n\n    Yields:\n        (Dict of resolved variables, Spec object)\n    \"\"\"\n    for resolved_vars, spec in _generate_variants(unresolved_spec):\n        assert not _unresolved_values(spec)\n        yield resolved_vars, spec\n\n\ndef grid_search(values):\n    \"\"\"Convenience method for specifying grid search over a value.\n\n    Arguments:\n        values: An iterable whose parameters will be gridded.\n    \"\"\"\n\n    return {\"grid_search\": values}\n\n\n_STANDARD_IMPORTS = {\n    \"random\": random,\n    \"np\": numpy,\n}\n\n_MAX_RESOLUTION_PASSES = 20\n\n\ndef resolve_nested_dict(nested_dict):\n    \"\"\"Flattens a nested dict by joining keys into tuple of paths.\n\n    Can then be passed into `format_vars`.\n    \"\"\"\n    res = {}\n    for k, v in nested_dict.items():\n        if isinstance(v, dict):\n            for k_, v_ in resolve_nested_dict(v).items():\n                res[(k, ) + k_] = v_\n        else:\n            res[(k, )] = v\n    return res\n\n\ndef format_vars(resolved_vars):\n    \"\"\"Formats the resolved variable dict into a single string.\"\"\"\n    out = []\n    for path, value in sorted(resolved_vars.items()):\n        if path[0] in [\"run\", \"env\", \"resources_per_trial\"]:\n            continue  # TrialRunner already has these in the experiment_tag\n        pieces = []\n        last_string = True\n        for k in path[::-1]:\n            if isinstance(k, int):\n                pieces.append(str(k))\n            elif last_string:\n                last_string = False\n                pieces.append(k)\n        pieces.reverse()\n        out.append(_clean_value(\"_\".join(pieces)) + \"=\" + _clean_value(value))\n    return \",\".join(out)\n\n\ndef flatten_resolved_vars(resolved_vars):\n    \"\"\"Formats the resolved variable dict into a mapping of (str -> value).\"\"\"\n    flattened_resolved_vars_dict = {}\n    for pieces, value in resolved_vars.items():\n        if pieces[0] == \"config\":\n            pieces = pieces[1:]\n        pieces = [str(piece) for piece in pieces]\n        flattened_resolved_vars_dict[\"/\".join(pieces)] = value\n    return flattened_resolved_vars_dict\n\n\ndef _clean_value(value):\n    if isinstance(value, float):\n        return \"{:.5}\".format(value)\n    else:\n        return str(value).replace(\"/\", \"_\")\n\n\ndef _generate_variants(spec):\n    spec = copy.deepcopy(spec)\n    unresolved = _unresolved_values(spec)\n    if not unresolved:\n        yield {}, spec\n        return\n\n    grid_vars = []\n    lambda_vars = []\n    for path, value in unresolved.items():\n        if callable(value):\n            lambda_vars.append((path, value))\n        else:\n            grid_vars.append((path, value))\n    grid_vars.sort()\n\n    grid_search = _grid_search_generator(spec, grid_vars)\n    for resolved_spec in grid_search:\n        resolved_vars = _resolve_lambda_vars(resolved_spec, lambda_vars)\n        for resolved, spec in _generate_variants(resolved_spec):\n            for path, value in grid_vars:\n                resolved_vars[path] = _get_value(spec, path)\n            for k, v in resolved.items():\n                if (k in resolved_vars and v != resolved_vars[k]\n                        and _is_resolved(resolved_vars[k])):\n                    raise ValueError(\n                        \"The variable `{}` could not be unambiguously \"\n                        \"resolved to a single value. Consider simplifying \"\n                        \"your configuration.\".format(k))\n                resolved_vars[k] = v\n            yield resolved_vars, spec\n\n\ndef _assign_value(spec, path, value):\n    for k in path[:-1]:\n        spec = spec[k]\n    spec[path[-1]] = value\n\n\ndef _get_value(spec, path):\n    for k in path:\n        spec = spec[k]\n    return spec\n\n\ndef _resolve_lambda_vars(spec, lambda_vars):\n    resolved = {}\n    error = True\n    num_passes = 0\n    while error and num_passes < _MAX_RESOLUTION_PASSES:\n        num_passes += 1\n        error = False\n        for path, fn in lambda_vars:\n            try:\n                value = fn(_UnresolvedAccessGuard(spec))\n            except RecursiveDependencyError as e:\n                error = e\n            except Exception:\n                raise ValueError(\n                    \"Failed to evaluate expression: {}: {}\".format(path, fn))\n            else:\n                _assign_value(spec, path, value)\n                resolved[path] = value\n    if error:\n        raise error\n    return resolved\n\n\ndef _grid_search_generator(unresolved_spec, grid_vars):\n    value_indices = [0] * len(grid_vars)\n\n    def increment(i):\n        value_indices[i] += 1\n        if value_indices[i] >= len(grid_vars[i][1]):\n            value_indices[i] = 0\n            if i + 1 < len(value_indices):\n                return increment(i + 1)\n            else:\n                return True\n        return False\n\n    if not grid_vars:\n        yield unresolved_spec\n        return\n\n    while value_indices[-1] < len(grid_vars[-1][1]):\n        spec = copy.deepcopy(unresolved_spec)\n        for i, (path, values) in enumerate(grid_vars):\n            _assign_value(spec, path, values[value_indices[i]])\n        yield spec\n        if grid_vars:\n            done = increment(0)\n            if done:\n                break\n\n\ndef _is_resolved(v):\n    resolved, _ = _try_resolve(v)\n    return resolved\n\n\ndef _try_resolve(v):\n    if isinstance(v, sample_from):\n        # Function to sample from\n        return False, v.func\n    elif isinstance(v, dict) and len(v) == 1 and \"eval\" in v:\n        # Lambda function in eval syntax\n        return False, lambda spec: eval(\n            v[\"eval\"], _STANDARD_IMPORTS, {\"spec\": spec})\n    elif isinstance(v, dict) and len(v) == 1 and \"grid_search\" in v:\n        # Grid search values\n        grid_values = v[\"grid_search\"]\n        if not isinstance(grid_values, list):\n            raise TuneError(\n                \"Grid search expected list of values, got: {}\".format(\n                    grid_values))\n        return False, grid_values\n    return True, v\n\n\ndef _unresolved_values(spec):\n    found = {}\n    for k, v in spec.items():\n        resolved, v = _try_resolve(v)\n        if not resolved:\n            found[(k, )] = v\n        elif isinstance(v, dict):\n            # Recurse into a dict\n            for (path, value) in _unresolved_values(v).items():\n                found[(k, ) + path] = value\n        elif isinstance(v, list):\n            # Recurse into a list\n            for i, elem in enumerate(v):\n                for (path, value) in _unresolved_values({i: elem}).items():\n                    found[(k, ) + path] = value\n    return found\n\n\nclass _UnresolvedAccessGuard(dict):\n    def __init__(self, *args, **kwds):\n        super(_UnresolvedAccessGuard, self).__init__(*args, **kwds)\n        self.__dict__ = self\n\n    def __getattribute__(self, item):\n        value = dict.__getattribute__(self, item)\n        if not _is_resolved(value):\n            raise RecursiveDependencyError(\n                \"`{}` recursively depends on {}\".format(item, value))\n        elif isinstance(value, dict):\n            return _UnresolvedAccessGuard(value)\n        else:\n            return value\n\n\nclass RecursiveDependencyError(Exception):\n    def __init__(self, msg):\n        Exception.__init__(self, msg)\n", "path": "python/ray/tune/suggest/variant_generator.py"}]}