non-zero coefficients are first selectedas the norm of the solutions increases by the time the norm reaches around all variables for which have been correctly identified and the remaining parameters are estimated as exactly only after the norm reaches around will these "spuriousparameters be estimated to be non-zero for this examplethe regularization parameter varied from - to - - norm figure regularization paths for lasso regression solutions as function of the norm of the solutions reproducing kernel hilbert spaces in this sectionwe formalize the idea outlined at the end of section of extending finite dimensional feature maps to those that are functions by introducing special type of hilbert space of functions known as reproducing kernel hilbert space (rkhsalthough the theory extends naturally to hilbert spaces of complex-valued functionswe restrict attention to hilbert spaces of real-valued functions here to evaluate the loss of learner in some class of functions gwe do not need to explicitly construct -ratherit is only required that we can evaluate at all the feature vectors xn of the training set defining property of an rkhs is that function evaluation at point can be performed by simply taking the inner product of with some feature function associated with we will see that this property becomes particularly useful in light of the representer theorem (see section )which states that the learner itself can be represented as linear combination of the set of feature functions { xi nconsequentlywe can evaluate learner at the feature vectors {xi by taking linear combinations of terms of the form (xi hk xi ig collecting these inner products into matrix [ (xi )ij (the gram matrix of the { xi })we will see that the feature vectors {xi only enter the loss minimization problem through