bayes optimal decision rule ( all classes are priori equally likelyand the likelihood functionone obtains the posterior pdf via bayesformula ( class is then assigned to feature vector according to the highest posterior probabilitythat iswe classify according to the bayes optimal decision ruleb argmax ( )( naive bayes which is exactly ( since the discrete density ( ) is usually not knownthe aim is to approximate it well with function ( xfrom some class of functions note that in this contextg(xrefers to discrete density ( probability mass functionfor given suppose feature vector [ ]of features has to be classified into one of the classes for examplethe classes could be different people and the features could be various facial measurementssuch as the width of the eyes divided by the distance between the eyesor the ratio of the nose height and mouth width in the naive bayes methodthe class of approximating functions is chosen such that ( yg( yg( )that isconditional on the labelall features are independent assuming uniform prior for ythe posterior pdf can thus be written as ( xp = ( )where the marginal pdfs ( ) belong to given class of approximating functions to classify xsimply take the that maximizes the unnormalized posterior pdf for instancesuppose that the approximating class is such that ( yn(uy ) the corresponding posterior pdf is then kx uy ( uy ) exp ( thxexp = where uy :[uy uyp ]and th :{ uc- collects all model parameters the probability ( thxis maximal when kx uy is minimal thus argminy kx uy is the classifier that maximizes the posterior probability that isclassify as when uy is closest to in euclidean distance of coursethe parameters (herethe {uy and are unknown and have to be estimated from the training data we can extend the above idea to the case where also the variance depends on the class and feature jas in the next example example (naive bayes classificationtable lists the means and standard deviations of normally distributed featuresfor different classes how should feature vector [ ]be classifiedthe posterior pdf is ( ( thx(sy sy sy )- exp = where th :{ } - = again collects all model parameters the (unscaledvalues for ( thx) are and - respectively hencethe feature vector should be classified as the code follows