How to know when to stop trainning a deep network? I've been training several auto encoders containing two GRUs as encoder and decoder during last year. It occurred to me that while training there were sequences of epochs which their loss did change just a little bit(less than 0.005 or so). Every time I took this as a sign that my model is not going to be better and stopped training. Now that I think about it I wonder what if I continued to train for more epochs and loss of model started to decrease again? Is there a factor to decide how long and how many epochs are enough to train recurrent networks?
Thats the whole point behind early stopping, don’t train the network to too small training error. It is always a bit tricky with early stopping but following framework might help.

When training, also output validation error

• Every time validation error improved, store a copy of the weights

• When validation error not improved for some time, stop

• Return the copy of the weights stored 

After early stopping of the first run, train a second run and reuse
validation data 
How to reuse validation data


Start fresh, train with both training data and validation data up to
the previous number of epochs
Start from the weights in the first run, train with both training data
and validation data until the validation loss &lt; the training loss at the
early stopping point