estimate the conditional pdf ( xfrom the training datausing only feature vectors in the neighborhood of in section we explain the support vector methodology for classificationthis is based on the same reproducing kernel hilbert space ideas that proved successful for regression analysis in section finallya versatile way to do both classification and regression is to use classification and regression trees this is the topic of neural networks (provide yet another way to perform classification classification metrics the effectiveness of classifier istheoreticallymeasured in terms of the risk ( )which depends on the loss function used fitting classifier to iid training data {(xi yi )}ni= is established by minimizing the training loss ` ( loss(yi (xi ) = ( over some class of functions as the training loss is often poor estimator of the risk the risk is usually estimated as in ( )using instead test set {( )}ni= that is independent of the training setas explained in section to measure the performance of classifier on training or test setit is convenient to introduce the notion of loss matrix consider classification problem with classifier gloss function lossand classes if an input feature vector is classified as (xwhen the observed class is ythe loss incurred isby definitionloss( , yconsequentlywe may identify the loss function with matrix [lossjk)jk { }for the indicator loss functionthe matrix has on the diagonal and everywhere else another useful matrix is the confusion matrixdenoted by mwhere the jk)-th element of counts the number of times thatfor the training or test datathe actual (observedclass is whereas the predicted class is table shows the confusion matrix of some dog/cat/possum classifier loss matrix confusion matrix table confusion matrix for three classes predicted actual dog cat possum dog cat possum we can now express the classifier performance ( in terms of and as [ mjk , ( where is the elementwise product of and note that for the indicator loss( is simply tr( )/nand is called the misclassification error the expression ( makes misclassification error it clear that both the counts and the loss are important in determining the performance of classifier