imposing the condition that ( ( for functions in will ensure that where the null space contains only linear functionsas we will see to see that this is in fact an rkhswe derive its reproducing kernel using integration by parts (or directly from the taylor expansion above)write (xz (sds ( ( sds ( ( )ds if is kernelthen by the reproducing property it must hold that (xhgk ih (sk (sds so that must satisfy (xs( )where :max{ thereforenoting that (xuhk ku ih we have (see exercise (xuz (xs (usmax{xumin{xu} min{xu} ds the last expression is cubic function with quadratic and cubic terms that misses the constant and linear monomials this is not surprising considering the taylor' theorem interpretation of function if we now take as the space of functions of the following form (having zero second derivative) xx [ ]then ( is exactly of the form ( as consequence of the representer theorem the optimal solution to ( is linear combination of piecewise cubic functionsg(xe ai (xi ( = such function is called cubic spline with knots (with one knot at each data point xi -so calledbecause the piecewise cubic function between knots is required to be "tied togetherat the knots the parameters ae are determined from ( for instance by solving ( with matrices [ (xi )]nij= and with -th row of the form [ xi for example (smoothing splinefigure shows various cubic smoothing splines for the data ( )( )( )( )( in the figurewe use the reparameterization /( gfor the smoothing parameter thus [ ]where means an infinite penalty for curvature (leading to the ordinary linear regression solutionand does not penalize curvature at all and leads to perfect fit via the so-called natural spline of course the latter will generally lead to overfitting for from up to the solutions will be close to the simple linear regression linewhile only for very close to the shape of the curve changes significantly cubic spline