egularization and ernel ethods the purpose of this is to familiarize the reader with two central concepts in modern data science and machine learningregularization and kernel methods regularization provides natural way to guard against overfitting and kernel methods offer broad generalization of linear models herewe discuss regularized regression (ridgelassoas bridge to the fundamentals of kernel methods we introduce reproducing kernel hilbert spaces and show that selecting the best prediction function in such spaces is in fact finite-dimensional optimization problem applications to spline fittinggaussian process regressionand kernel pca are given introduction in this we return to the supervised learning setting of (regressionand expand its scope given training data {( )(xn yn )}we wish to find prediction function (the learnergt that minimizes the (squared-errortraining loss (yi (xi )) ` (gn = within class of functions as noted in if is the set of all possible functions then choosing any function with the property that (xi yi for all will give zero training lossbut will likely have poor generalization performance (that issuffer from overfittingrecall from theorem that the best possible prediction function (over all gfor the squared-error risk ( ( )) is given by (xe[ xthe class should be simple enough to permit theoretical understanding and analysis butat the same timerich enough to contain the optimal function (or function close to gthis ideal can be realized by taking to be hilbert space ( complete inner product spaceof functionssee appendix many of the classes of functions that we have encountered so far are in fact hilbert spaces in particularthe set of linear functions on is hilbert space to see this hilbert space