optimal decision boundary thereforea feature vector is classified according to or - depending on whether gt ( or respectively the optimal decision boundary is given by the set of for which gt ( similar to the cubic smoothing spline or rkhs setting in ( )we can consider finding the best classifiergiven the training datavia the penalized goodness-of-fit optimizationn [ yi (xi )] kgk min ghh = for some regularization parameter it will be convenient to define : ne and to solve the equivalent problem min [ yi (xi )]kgk ghh = we know from the representer theorem that if is the reproducing kernel corresponding to hthen the solution is of the form (assuming that the null space has constant term only) (xa ai (xi ( = substituting into the minimization expression yields the analogue of ( ) min [ yi ( {ka} )]akaa, = ( where is the gram matrix this is convex optimization problemas it is the sum of convex quadratic and piecewise linear term in defining li :gai /yi and :[ ln ]we show in exercise that the optimal and in ( can be obtained by solving the "dualconvex optimization problem max subject ton = li xx li yi (xi = = ( and = ai (xi for any for which ( in view of ( )the optimal prediction function (pre-classifiergt is then given by gt (xa ai (xi xa yi li (xi xg = = ( to mitigate possible numerical problems in the calculation of it is customary to take an overall averagen ( |jjj where : ( ) =