nearest neighbor classification with this dataseteven if we just try to separate two classes using the previous methodwe do not get very good results let me introduce thereforea new classifierthe nearest neighbor classifier if we consider that each example is represented by its features (in mathematical termsas point in -dimensional space)we can compute the distance between examples we can choose different ways of computing the distancefor exampledef distance( )'computes squared euclidean distancereturn np sum( - )** now when classifyingwe adopt simple rulegiven new examplewe look at the dataset for the point that is closest to it (its nearest neighborand look at its labeldef nn_classify(training_settraining_labelsnew_example)dists np array([distance(tnew_examplefor in training_set]nearest dists argmin(return training_labels[nearestin this caseour model involves saving all of the training data and labels and computing everything at classification time better implementation would be to actually index these at learning time to speed up classificationbut this implementation is complex algorithm nownote that this model performs perfectly on its training datafor each pointits closest neighbor is itselfand so its label matches perfectly (unless two examples have exactly the same features but different labelswhich can happenthereforeit is essential to test using cross-validation protocol using ten folds for cross-validation for this dataset with this algorithmwe obtain percent accuracy as we discussed in the earlier sectionthe cross-validation accuracy is lower than the training accuracybut this is more credible estimate of the performance of the model we will now examine the decision boundary for thiswe will be forced to simplify and look at only two dimensions (just so that we can plot it on paper