training deep leaning models regularizing the model from the previous exampleis easy to see that while fitting modelsa central problem is to get the capacity of the model exactly right so that one neither overfits nor underfits the data regularization can be simply seen as any modification to the model (or its training processthat intends to improve the error on the unseen data (at the cost of the error on the training databy systematically limiting the capacity of the model this of process systematically limiting or regulating the capacity of the model is guided by portion of the labelled data that is not used of training this data is commonly referred to as the validation set in our running examplea regularized version of least squares takes the form bxwhere is vector such that is minimizedand is user-defined parameter that controls the complexity hereby introducing the term we are penalizing models with extra capacity to see why this is the caseconsider fitting least squares model using polynomial of degree but the values in the vector have zeros and non-zeros as opposed to thisconsider the case where all values in the vector are non-zeros for all practical purposesthe former model is model with degree and lower value of the term allows us to balance accuracy over the training data with the complexity of the model lower values of imply model with lower capacity one natural question to ask is why we do not simply use the validation set as guide and increase the degree of the polynomial in the previous example since the degree of the polynomial is proxy for the capacity of the modelwhy can' we use that to tune the model capacitywhy do we need to introduce the change in the model instead of previously)the answer is that we want to systematically limit the capacity of the model for which we need fine-grained control changing the model capacity by varying the degree of the model is very coarse-graineddiscrete knobwhile varying is very fine grained