penalized regression the important variations of ols regression fall under the theme of penalized regression in ordinary regressionthe returned fit is the best fit on the training datawhich can lead to overfitting penalizing means that we add penalty for overconfidence in the parameter values penalized regression is about tradeoffs penalized regression is another example of the bias-variance tradeoff when using penaltywe get worse fit in the training data as we are adding bias on the other handwe reduce the variance and tend to avoid overfitting thereforethe overall result might be generalized in better way and penalties there are two types of penalties that are typically used for regressionl and penalties the penalty means that we penalize the regression by the sum of the absolute values of the coefficientsand the penalty penalizes by the sum of squares let us now explore these ideas formally the ols optimization is given as followsin the preceding formulawe find the vector that results in the minimum squared distance to the actual target when we add an penaltywe instead optimize the following formulaherewe are trying to simultaneously make the error smallbut also make the values of the coefficients small (in absolute termsusing penalty means that we use the following formulathe difference is rather subtlewe now penalize by the square of the coefficient rather than its absolute value howeverthe difference in the results is dramatic