training set score test set score this discrepancy between performance on the training set and the test set is clear sign of overfittingand therefore we should try to find model that allows us to control complexity one of the most commonly used alternatives to standard linear regression is ridge regressionwhich we will look into next ridge regression ridge regression is also linear model for regressionso the formula it uses to make predictions is the same one used for ordinary least squares in ridge regressionthoughthe coefficients (ware chosen not only so that they predict well on the training databut also to fit an additional constraint we also want the magnitude of coefficients to be as small as possiblein other wordsall entries of should be close to zero intuitivelythis means each feature should have as little effect on the outcome as possible (which translates to having small slope)while still predicting well this constraint is an example of what is called regularization regularization means explicitly restricting model to avoid overfitting the particular kind used by ridge regression is known as regularization ridge regression is implemented in linear_model ridge let' see how well it does on the extended boston housing datasetin[ ]from sklearn linear_model import ridge ridge ridge(fit(x_trainy_trainprint("training set score{ }format(ridge score(x_trainy_train))print("test set score{ }format(ridge score(x_testy_test))out[ ]training set score test set score as you can seethe training set score of ridge is lower than for linearregressionwhile the test set score is higher this is consistent with our expectation with linear regressionwe were overfitting our data ridge is more restricted modelso we are less likely to overfit less complex model means worse performance on the training setbut better generalization as we are only interested in generalization performancewe should choose the ridge model over the linearregression model mathematicallyridge penalizes the norm of the coefficientsor the euclidean length of supervised machine learning algorithms