training deep leaning models we can transform to be vector of values [ that isif it will be transformed to [ following this transformationwe can generate least squares model using the preceding formula under the hoodwe are approximating the given data with second order polynomial (degree equationand the least squares algorithm is simply curve fitting or generating the coefficients for each of [ similarlywe can generate another model with the least squares algorithmbut we will transform to [ that iswe are approximating the given data with polynomial with degree by increasing the degree of the polynomialwe can fit arbitrary data it is easy to see that if we have data pointsa polynomial of degree can perfectly fit the data it is also easy to see that such model is simply memorizing the data we can use this example to develop perspective on model capacityoverfittingand underfitting the degree of the polynomial we use to fit the data is basically proxy for the capacity of the model the greater the degreethe higher is the capacity of the model let us assume that the data were generated using polynomial of degree with some noise alsonote that while fitting the datawe do not know anything about the process that generated the data we have to produce model that best fits the data essentiallywe do not know how much of the data is the pattern and how much of the data is noise on such datasetif we use models with high enough capacity (degree of the polynomial greater than in the worst case equal to the number of data points)we can get perfect model when evaluated on the training datahoweverthis model will do very poorly on unseen databecause it has essentially fit the noise this is overfitting if we use model with low capacity (less than )it will fit neither the training data nor the unseen data well this is underfitting