title: Hyperparameter optimization for Deep Learning Structures using Bayesian Optimization
body: <p>I have constructed a CLDNN (Convolutional, LSTM, Deep Neural Network) structure for raw signal classification task.</p>
  
  <p>Each training epoch runs for about 90 seconds and the hyperparameters seems to be very difficult to optimize.</p>
  
  <p>I have been research various ways to optimize the hyperparameters (e.g. random or grid search) and found out about Bayesian Optimization.</p>
  
  <p>Although I am still not fully understanding the optimization algorithm, I feed like it will help me greatly.</p>
  
  <p>I would like to ask few questions regarding the optimization task.</p>
  
  <ol>
  <li>How do I set up the Bayesian Optimization with regards to a deep network?(What is the cost function we are trying to optimize?)</li>
  <li>What is the function I am trying to optimize? Is it the cost of the validation set after N epochs?</li>
  <li>Is spearmint a good starting point for this task? Any other suggestions for this task?</li>
  </ol>
  
  <p>I would greatly appreciate any insights into this problem.</p>