the data science process (if not the most popularprogramming languages for data science for more informationsee model diagnostics and model comparison you'll be building multiple models from which you then choose the best one based on multiple criteria working with holdout sample helps you pick the best-performing model holdout sample is part of the data you leave out of the model building so it can be used to evaluate the model afterward the principle here is simplethe model should work on unseen data you use only fraction of your data to estimate the model and the other partthe holdout sampleis kept out of the equation the model is then unleashed on the unseen data and error measures are calculated to evaluate it multiple error measures are availableand in figure we show the general idea on comparing models the error measure used in the example is the mean square error mse --(yi = figure formula for mean square error mean square error is simple measurecheck for every prediction how far it was from the truthsquare this errorand add up the error of every prediction figure compares the performance of two models to predict the order size from the price the first model is size price and the second model is size to train size price predicted model predicted model error model error model - test total figure holdout sample helps you compare models and ensures that you can generalize results to data that the model has not yet seen