kernel trick optimization problem the reason is that any solution to ( can be represented as finite-dimensional linear combination of kernel functionsevaluated at the training sample this is known as the kernel trick theorem representer theorem the solution to the penalized optimization problem ( is of the form (xn ai (xi xi= ( )( = where { qm is basis of prooflet span xi clearlyf thenthe hilbert space can be represented as where is the orthogonal complement of in other wordsf is the class of functions ih xi ih iit followsby the reproducing kernel propertythat for all (xi xi ih nowtake any and write it as with and by the definition of the null space we have kgk moreoverby pythagorastheoremthe latter is equal to it follows that loss(yi (xi )gkgk loss(yi (xi (xi ) = = loss(yi (xi (xi ) = since we can obtain equality by taking this implies that the minimizer of the penalized optimization problem ( lies in the subspace of and hence is of the form ( substituting the representation ( of into ( gives the finite-dimensional optimization problemn min loss(yi (ka qe) akaarn erm = ( where is the (grammatrix with entries [ (xi ) nj nq is the matrix with entries [ (xi ) nj