training deep leaning models analysis of overfitting and underfitting the ideal goal of the iterative cycle of model improvement is to develop model where the performance over the training set and validation set is nearly equal to the established performance limit (proxy for bayes errorfigure - illustrates this final destination of the model improvement process while iteratively developing new modelshoweverthe practitioner will encounter underfitting and overfitting underfitting occurs when the model' performance over the training and validation set is nearly equal but the performance is below the desired level this is an outcome of poorly developed model in which the parameters have not appropriately captured the patterns within the training data on the other handoverfitting occurs when the model performance over the validation set is significantly lower than its performance over the training set this is direct outcome of model that has learned far too many complicated patterns that should have ideally been considered as noise such model (which accounted for noise in the data as valid patternsdelivers top performance on the training (seendata but performs poorly on unseen data underfitting and overfitting are not mutually exclusive in scenario where model is underfittingwe more formally define this situation as model with high bias similarlywhen model that has learned several complex patterns from noise delivers highly inconsistent performance on unseen datawe say the model has high variance ideallywe would need model that has low bias and low variance detecting whether the model is overfitting or underfitting is the first step after new model is trained in the case of underfittingthe key step is to increase the effective capacity of the modelwhich is typically done by modifying the architecture (increasing layerswidthsand so forthin the case of overfittingthe key steps are either regularization methods (covered later in this or increasing the dataset size an important visualization is learning curveswhich plot performance metrics on the axis and the training data made available to the model on the axis this is quite useful in determining whether an investment in procuring more labelled data makes sense