polyregpress py import numpy as np import matplotlib pyplot as plt def generate_data (beta sig ) np random rand( *np arange ( beta reshape ( , sig np random randn ( )return uy np random seed ( beta np array ([[ - - ]]tsig = ** , generate_data (beta ,sig ,nx np ones (( ) maximum number of parameters press np zeros ( + for in range ( , )if np hstack ((xu**( - ))add column to matrix np linalg pinv(xprojection matrix press [knp sum (( /( np diag(preshape ( , )))** plt plotpress [ : ]/nthe press values divided by for the constantlinearquadraticcubicand quartic order polynomial regression models arerespectively and hencethe cubic polynomial regression model has the lowest pressindicating that it has the best predictive performance in-sample risk and akaike information criterion in section we introduced the in-sample risk as measure for the accuracy of the prediction function to recapitulategiven fixed data set with associated response vector and matrix of explanatory variables xthe in-sample risk of prediction function is defined as `in ( :ex loss(yg( ))( where ex signifies that the expectation is taken under different probability modelin which takes the values xn with equal probabilityand given xi the random variable is drawn from the conditional pdf ( xi the difference between the in-sample risk and the training loss is called the optimism for the squared-error losstheorem expresses the expected optimism of learner gt as two times the average covariance between the predicted values and the responses if the conditional variance of the error (xgiven does not depend on xthen the expected in-sample risk of learner gt averaged over all training setshas simple expression