introduction (xx xwith respect to ( )for every fixed in other wordstake (xto be equal to the class label for which [ xis maximal bayes error rate the formulation ( allows for "ties"when there is an equal probability between optimal classes for feature vector assigning one of these tied classes arbitrarily (or randomlyto does not affect the loss function and so we assume for simplicity that (xis always scalar value note thatas was the case for the regression (seee theorem )the optimal prediction function depends on the conditional pdf ( xp[ xhoweversince we assign to class if ( xf ( xfor all zwe do not need to learn the entire surface of the function ( )we only need to estimate it well enough near the decision boundary { ( xf ( )for any choice of classes and this is because the assignment ( divides the feature space into regionsry { ( xmaxz ( )} recall that for any supervised learning problem the smallest possible expected loss (that isthe irreducible riskis given by ``(gfor the indicator lossthe irreducible risk is equal to [ ( )this smallest possible probability of misclassification is often called the bayes error rate for given training set ta classifier is often derived from pre-classifier gt which is prediction function (learnerthat can take any real valuerather than only values in the set of class labels typical situation is the case of binary classification with labels - and where the prediction function gt is function taking values in the interval [- and the actual classifier is given by sign(gt it will be clear from the context whether prediction function gt should be interpreted as classifier or pre-classifier the indicator loss function may not always be the most appropriate choice of loss function for given classification problem for examplewhen diagnosing an illnessthe mistake in misclassifying person as being sick when in fact the person is healthy may be less serious than classifying the person as healthy when in fact the person is sick in section we consider various classification metrics there are many ways to fit classifier to training set {( )(xn yn )the approach taken in section is to use bayesian framework for classification here the conditional pdf ( xis viewed as posterior pdf ( xf ( yf (yfor given class prior (yand likelihood ( ysection discusses linear and quadratic discriminant analysis for classificationwhich assumes that the class of approximating functions for the conditional pdf ( yis parametric class of gaussian densities as result of this choice of gthe marginal (xis approximated via gaussian mixture density in contrastin the logistic or soft-max classification in section the conditional pdf ( xis approximated using more flexible class of approximating functions as result of thisthe approximation to the marginal density (xdoes not belong to simple parametric class (such as gaussian mixtureas in unsupervised learningthe crossentropy loss is the most common choice for training the learner the -nearest neighbors methoddiscussed in section is yet another approach to classification that makes minimal assumptions on the class here the aim is to directly