lassification the purpose of this is to explain the mathematical ideas behind well-known classification techniques such as the naive bayes methodlinear and quadratic discriminant analysislogistic/softmax classificationthe -nearest neighbors methodand support vector machines introduction classification methods are supervised learning methods in which categorical response variable takes one of possible values (for example whether person is sick or healthy)which is to be predicted from vector of explanatory variables (for examplethe blood pressureageand smoking status of the person)using prediction function in this senseg classifies the input into one of the classessay in the set { for this reasonwe will call classification function or simply classifier as with any supervised learning technique (see section )the goal is to minimize the expected loss or risk `(ge loss(yg( )classifier ( for some loss functionloss( , )that quantifies the impact of classifying response via (xthe natural loss function is the zero-one (also written - or indicator lossloss( , : { }that isthere is no loss for correct classification ( yand unit loss for misclassification ( yin this case the optimal classifier is given in the following theorem indicator loss theorem optimal classifier for the loss function loss( , { }an optimal classification function is (xargmax [ xy{ , - ( proofthe goal is to minimize `(ge { ( )over all functions taking values in { conditioning on givesby the tower property`(ge ( [ (xx)and so minimizing `(gwith respect to can be accomplished by maximizing [