introduction to machine learning and deep learning we now consider the important concept of model capacitywhich corresponds to the degree of the polynomial in this example the data we generated was using second order polynomial (degree with some noise thenwe tried to approximate the data using three models (of degrees and respectivelythe higher the degreethe more expressive is the model--that isit can accommodate more variation this ability to accommodate variation corresponds to the notion of model capacity that iswe say that the model with degree has higher capacity than the model with degree whichin turnhas higher capacity than the model with degree isn' having higher capacity always good thingit turns out that it is notwhen we consider that all real-world datasets contain some noise and higher capacity model will end up in just fitting the noise in addition to the signal in the data this is why we observe that the model with degree does better on the unseen data as compared to the model with degree in this examplewe knew how the data was generated (with second order polynomial (degree with some noise)hencethis observation is quite trivial howeverin the real worldwe don' know the underlying mechanism by which the data is generated this leads us to the fundamental challenge in machine learningdoes the model truly generalizeand the only true test for that is the performance over unseen data in sensethe concept of capacity corresponds to the simplicity or parsimony of the model model with high capacity can approximate more complex data this is how many how many free variablescoefficients the model has in our examplethe model with degree does not have capacity sufficient to approximate the data this is commonly referred to as underfitting correspondinglythe model with degree has extra capacity and overfits the data as thought experimentconsider what would happen if we had model with degree given that we had data points as training datawe would have an -degree polynomial that would perfectly approximate the data this is the ultimate pathological case where in there is no learning at all the model has coefficients and can simply