rules that work well on this dataremember that we are not interested in making predictions for this datasetwe already know the answers for these customers we want to know if new customers are likely to buy boat we therefore want to find rule that will work well for new customersand achieving percent accuracy on the training set does not help us there we might not expect that the rule our data scientist came up with will work very well on new customers it seems too complexand it is supported by very little data for examplethe "or is not divorcedpart of the rule hinges on single customer the only measure of whether an algorithm will perform well on new data is the evaluation on the test set howeverintuitively we expect simple models to generalize better to new data if the rule was "people older than want to buy boat,and this would explain the behavior of all the customerswe would trust it more than the rule involving children and marital status in addition to age thereforewe always want to find the simplest model building model that is too complex for the amount of information we haveas our novice data scientist didis called overfitting overfitting occurs when you fit model too closely to the particularities of the training set and obtain model that works well on the training set but is not able to generalize to new data on the other handif your model is too simple--say"everybody who owns house buys boat"--then you might not be able to capture all the aspects of and variability in the dataand your model will do badly even on the training set choosing too simple model is called underfitting the more complex we allow our model to bethe better we will be able to predict on the training data howeverif our model becomes too complexwe start focusing too much on each individual data point in our training setand the model will not generalize well to new data there is sweet spot in between that will yield the best generalization performance this is the model we want to find the trade-off between overfitting and underfitting is illustrated in figure - and also provablywith the right math supervised learning