title: How to avoid overfitting with keras?
body: <pre><code>def build_model():
    model = keras.models.Sequential()
  
    model.add(keras.layers.Flatten(input_shape=[32,32,3]))
    keras.layers.Dropout(rate=0.2)
  
    model.add(keras.layers.Dense(500, activation=&quot;relu&quot;))
    keras.layers.Dropout(rate=0.2)
  
    model.add(keras.layers.Dense(300, activation=&quot;relu&quot;))
    keras.layers.Dropout(rate=0.2)  
  
    model.add(keras.layers.Dense(10, activation=&quot;softmax&quot;))
    model.compile(loss='sparse_categorical_crossentropy', optimizer=keras.optimizers.SGD(), metrics=['accuracy'])
    return model 
  
  keras_clf = keras.wrappers.scikit_learn.KerasClassifier(build_model)
  
  def exponential_decay_fn(epoch): 
    return 0.05 * 0.1**(epoch / 20)
  
  lr_scheduler = keras.callbacks.LearningRateScheduler(exponential_decay_fn)
  
  history = keras_clf.fit(np.array(X_train_new), np.array(y_train_new), epochs=100,
                        validation_data=(np.array(X_validation), np.array(y_validation)),
                        callbacks=[keras.callbacks.EarlyStopping(patience=10),lr_scheduler])
  </code></pre>
  <p><a href="https://i.stack.imgur.com/RM6Cb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RM6Cb.png" alt="The result of the above code" /></a></p>
  <p>I use 'drop out', 'early stopping', and 'lr scheduler'. The results seem overfitting, I tried to reduce n_neurons of hidden layers to (300, 100). The results were underfitting, the accuracy of the train set was only around 0.5.</p>
  <p>Are there any suggestions?</p>