title: Hyperopt: defining search space
body: <p>I'm having trouble with the logic of the search space definition.</p>
  
  <p>I want to search over these:</p>
  
  <ul>
  <li>The type of model to use (features_and_hours, features_only, hours_only, no_features_no_hours)</li>
  <li>The number of hidden units (output_units)</li>
  <li>regularization for the kernel matrix (type = l1, l2, or l1l2)</li>
  <li>regularization values for the kernel matrix (anywhere from 0.0 to 0.5)</li>
  <li>regularization for the activities (type = l1, l2, or l1l2)</li>
  <li>regularization values for the activities (anywhere from 0.0 to 0.5)</li>
  <li>number of epochs (num_epochs, either 1, 5, or 10)</li>
  <li>optimizer to be used (either adadelta, adam, or rmsprop)</li>
  <li>whether and how to apply attention (before, after, or none)</li>
  </ul>
  
  <p>Here's the way I've set this up, following <a href="https://github.com/keras-team/keras/issues/1591" rel="nofollow noreferrer">this example</a> (second post on page, by jacobzweig)</p>
  
  <pre><code>def para_space():
  
          space_paras = {'model_type': hp.choice('model_type', ['features_and_hours', 'features_only', 'hours_only', 'no_features_no_hours']),
                          'output_units': hp.uniform('output_units', 1, 10),
                          'kernel_reg': hp.choice('kernel_reg', [{'reg_type':'l1', 'reg_vals': hp.uniform('reg_vals', 0.0, 0.5)},
                                                                  {'reg_type':'l2','reg_vals': hp.uniform('reg_vals', 0.0, 0.5)},
                                                                  {'reg_type':'l1l2', 'reg_vals': hp.uniform('reg_vals', 0.0, 0.5)}]),
                          'activity_reg': hp.choice('activity_reg', [{'reg_type':'l1', 'reg_vals': hp.uniform('reg_vals', 0.0, 0.5)},
                                                                  {'reg_type':'l2','reg_vals': hp.uniform('reg_vals', 0.0, 0.5)},
                                                                  {'reg_type':'l1l2', 'reg_vals': hp.uniform('reg_vals', 0.0, 0.5)}]),
                          'num_epochs': hp.choice('num_epochs', [1, 5, 10]),
                          'optimizer': hp.choice('optimizer', ['adadelta', 'adam', 'rmsprop']),
                          'attention': hp.choice('attention', ['before', 'after', 'none'])}
          return space_paras
  </code></pre>
  
  <p>The error I'm getting is:</p>
  
  <pre><code>Using TensorFlow backend.
  Traceback (most recent call last):
    File "Jan22Model1.py", line 374, in &lt;module&gt;
      best = fmin(lstm_model_1, params, algo=tpe.suggest, max_evals=5, trials=trials)
    File "/user/pkgs/anaconda2/lib/python2.7/site-packages/hyperopt/fmin.py", line 307, in fmin
      return_argmin=return_argmin,
    File "/user/pkgs/anaconda2/lib/python2.7/site-packages/hyperopt/base.py", line 635, in fmin
      return_argmin=return_argmin)
    File "/user/pkgs/anaconda2/lib/python2.7/site-packages/hyperopt/fmin.py", line 314, in fmin
      pass_expr_memo_ctrl=pass_expr_memo_ctrl)
    File "/user/pkgs/anaconda2/lib/python2.7/site-packages/hyperopt/base.py", line 776, in __init__
      raise DuplicateLabel(label)
  hyperopt.exceptions.DuplicateLabel: reg_vals
  </code></pre>
  
  <p>But in the example, there seem to be duplicate labels that don't throw an error. What am I doing wrong?</p>