How to know when to stop trainning a deep network?
Thats the whole point behind early stopping, don’t train the network to too small training error. It is always a bit tricky with early stopping but following framework might help.

When training, also output validation error

• Every time validation error improved, store a copy of the weights

• When validation error not improved for some time, stop

• Return the copy of the weights stored 

After early stopping of the first run, train a second run and reuse
validation data 
How to reuse validation data


Start fresh, train with both training data and validation data up to
the previous number of epochs
Start from the weights in the first run, train with both training data
and validation data until the validation loss &lt; the training loss at the
early stopping point