What to do after selecting a model with cross-validation? I have been building a neural network for classification.

To select my best model. I have been using 10-Fold cross validation. and selected the network that gives the highest mean accuracy.

Now that I have selected the best model, I want to use all the data I have to train this model because the amount of data I have is limited (I will merge training, dev and test data). 

My issue is that, when training with all the data, I don't know when to stop training. Training loss is not an indicator for sure. Usually, I have a development set that I use to monitor training. When the training loss does not improve anymore, I stop training. 

Any suggestions on how to supervise a model with only training data? In other words, how to tell when the network needs to stop?
When 'overtraining' is not a problem (as in it will not diverge if you use more time), just use all your data and the empirically found optimal hyper parameters. In case of neural networks this is not the case (although in my experience, a lot of architectures converge to a specific test error, and take a long time to end up diverging again). I see a few options that you could try:


Most obvious one is keeping a (small) validation set around to use as indicator for early stopping (don't think of this as throwing away data, you still use it to train your network better)
Use same weight initalization as one of your folds and run for the same amount of epochs, same initialization should make convergence rate more similar than new random initialization
Keep all the cross validation models and use them in an ensemble instead of retraining the full model