egression many supervised learning techniques can be gathered under the name "regressionthe purpose of this is to explain the mathematical ideas behind regression models and their practical aspects we analyze the fundamental linear model in detailand also discuss nonlinear and generalized linear models introduction francis galton observed in an article in that the heights of adult offspring areon the wholemore "averagethan the heights of their parents galton interpreted this as degenerative phenomenonusing the term "regressionto indicate this "return to mediocritynowadaysregression refers to broad class of supervised learning techniques where the aim is to predict quantitative response (outputvariable via function (xof an explanatory (inputvector [ ]consisting of featureseach of which can be continuous or discrete for instanceregression could be used to predict the birth weight of baby (the response variablefrom the weight of the motherher socio-economic statusand her smoking habits (the explanatory variableslet us recapitulate the framework of supervised learning established in the aim is to find prediction function that best guesses what the random output will be for random input vector the joint pdf (xyof and is unknownbut training set {( )(xn yn )is availablewhich is thought of as the outcome of random training set {( )(xn yn )of iid copies of (xyonce we have selected loss function loss( , )such as the squared-error loss loss( , ( ) recall the mnemonic use of "gfor "guess squared-error loss ( then the "bestprediction function is defined as the one that minimizes the risk `(ge loss(yg( )we saw in section that for the squared-error loss this optimal prediction function is the conditional expectation (xe[ xregression risk