note thatfrom ( )the optimal pre-classifier (xand the classifier sign (xonly depend on vectors xi for which li these vectors are called the support vectors of the support vector machine it is also important to note that the quadratic function in ( depends on the regularization parameter by defining ni :li /gi nwe can rewrite ( as min subject ton ni ni yi (xi ij = ni yi ni / = ( = for perfectly separable datathat isdata for which an affine plane can be drawn to perfectly separate the two classeswe may take as explained below otherwisec needs to be chosen via cross-validation or test data setfor example geometric interpretation for the linear kernel function (xx xx we have gt (xb bxp with and - ni= li yi xi ni= ai xi and so the decision boundary is an affine plane the situation is illustrated in figure the decision boundary is formed by the points such that gt ( the two sets { gt ( - and { gt ( are called the margins the distance from the points on margin to the decision boundary is /kbk figure classifying two classes (red and blueusing svm based on the "multipliers{li }we can divide the training samples {(xi yi )into three categories (see exercise )points for which li ( these are the support vectors on the margins (green encircled in the figureand are correctly classified support vectors