deep learning for text and sequences you're already taking basic steps to mitigate overfittingsuch as using dropoutas long as you aren' overfitting too badlyyou're likely under capacity increasing network capacity is typically done by increasing the number of units in the layers or adding more layers recurrent layer stacking is classic way to build more-powerful recurrent networksfor instancewhat currently powers the google translate algorithm is stack of seven large lstm layers--that' huge to stack recurrent layers on top of each other in kerasall intermediate layers should return their full sequence of outputs ( tensorrather than their output at the last timestep this is done by specifying return_sequences=true listing training and evaluat ing dropout-regularizedst acked gru model from keras models import sequential from keras import layers from keras optimizers import rmsprop model sequential(model add(layers gru( dropout= recurrent_dropout= return_sequences=trueinput_shape=(nonefloat_data shape[- ]))model add(layers gru( activation='relu'dropout= recurrent_dropout= )model add(layers dense( )model compile(optimizer=rmsprop()loss='mae'history model fit_generator(train_gensteps_per_epoch= epochs= validation_data=val_genvalidation_steps=val_stepsfigure shows the results you can see that the added layer does improve the results bitthough not significantly you can draw two conclusionsbecause you're still not overfitting too badlyyou could safely increase the size of your layers in quest for validation-loss improvement this has non-negligible computational costthough adding layer didn' help by significant factorso you may be seeing diminishing returns from increasing network capacity at this point licensed to