title: How to apply Dropout in GridSearchCV
body: <p>I use the following code to tune the hyperparameters (hidden layers, hidden neurons, batch size, optimizer) of an ANN.</p>
  <pre><code>## Part 2 - Tuning the ANN
  from keras.wrappers.scikit_learn import KerasRegressor
  from sklearn.model_selection import GridSearchCV
  from keras.models import Sequential
  from keras.layers import Dense
  def build_regressor(hidden_nodes, hidden_layers, optimizer):
    regressor = Sequential()
    regressor.add(Dense(units = hidden_nodes, kernel_initializer = 'uniform', activation = 'relu', input_dim = 7))
    for layer_size in range(hidden_layers):
        regressor.add(Dense(hidden_nodes, kernel_initializer = 'uniform', activation = 'relu'))
    regressor.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'linear'))
    regressor.compile(optimizer = optimizer, loss = 'mse', metrics = ['mse'])
    return regressor
  regressor = KerasRegressor(build_fn = build_regressor, epochs = 100)
  
  # Create a dictionary of tuning parameters
  parameters = {'hidden_nodes': list(range(2,101)), 'hidden_layers': [4,5,6,7], 'batch_size': [25,32], 'optimizer' : ['adam', 'nadam','RMSprop', 'adamax']}
  grid_search = GridSearchCV(estimator = regressor, param_grid = parameters, scoring = 'neg_mean_squared_error', cv = 10, n_jobs = 4)
  
  start = time.time()
  grid_search = grid_search.fit(X_train, y_train)
  end = time.time()
  elapsed = (end - start)/3600
  
  </code></pre>
  <p>Now I want to add a droppout layer after each of the hidden layer like this:</p>
  <pre><code>regressor1 = Sequential()
  regressor1.add(Dense(units = 41, kernel_initializer = 'uniform', activation = 'relu', input_dim = 7))
  regressor1.add(Dropout(0.1))
  regressor1.add(Dense(units = 41, kernel_initializer = 'uniform', activation = 'relu'))
  regressor1.add(Dropout(0.1))
  regressor1.add(Dense(units = 41, kernel_initializer = 'uniform', activation = 'relu'))
  regressor1.add(Dropout(0.1))
  regressor1.add(Dense(units = 41, kernel_initializer = 'uniform', activation = 'relu'))
  regressor1.add(Dropout(0.1))
  regressor1.add(Dense(units = 41, kernel_initializer = 'uniform', activation = 'relu'))
  regressor1.add(Dropout(0.1))
  regressor1.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'linear'))
  regressor1.compile(optimizer = 'nadam', loss = 'mse', metrics = ['mse'])
  history = regressor1.fit(X_train, y_train, batch_size = 25, epochs = 500, validation_data = (X_test, y_test), callbacks = [EarlyStopping(patience = 10)])
  
  </code></pre>
  <p>Is there a way to tune the number of dropout layers (the same number of hidden layers) and the dropout rates together with my current code?</p>
  <p>Thank you so much in advance,</p>