training deep leaning models early stopping one of the simplest techniques for regularization in deep learning is early stopping given training set and validation set and network with sufficient capacitywe observe that with increasing training stepsfirst both the error on the training set and validation set decreasesthen the error of the training set continues to decrease while the error in validation increases (see figure - the key idea with early stopping is to keep track of the model parameters/weights that give the best performance over the validation setand then to stop the training after this best performance so far over the validation set does not improve over predefined number of training steps figure - early stopping early stopping acts as regularizer by restricting the values the parameters/weights of the model can take (see figure - early stopping limits to neighborhood around the starting values (around soif we stop at wsthe values of ws are not possible this essentially restricts the capacity of the model