knn does poorly when the number of observations and features grow knn becomes computationally inefficient in this highly populatedhigh-dimensional space since it needs to calculate distances from the new point to many nearby labeled points in order to predict labels it cannot rely on an efficient model with reduced number of parameters to make the necessary prediction alsoknn is very sensitive to the choice of when is set too lowknn can overfitand when is set too highknn can underfit applications knn is regularly used in recommender systemssuch as those used to predict taste in movies (netflix)music (spotify)friends (facebook)photos (instagram)search (google)and shopping (amazonfor exampleknn can help predict what user will like given what similar users like (known as collaborative filteringor what the user has liked in the past (known as content-based filteringtree-based methods instead of using linear methodwe can have the ai build decision tree where all the instances are segmented or stratified into many regionsguided by the labels we have once this segmentation is completeeach region corresponds to particular class of label (for classification problemsor range of predicted values (for regression problemsthis process is similar to having the ai build rules automatically with the explicit goal of making better decisions or predictions single decision tree the simplest tree-based method is single decision treein which the ai goes once through the training datacreates rules for segmenting the data guided by the labelsand uses this tree to make predictions on the never-before-seen validation or test set howevera single decision tree is usually poor at generalizing what it has learned during training to never-before-seen cases because it usually overfits the training data during its one and only training iteration bagging to improve the single decision treewe can introduce bootstrap aggregation (more commonly known as bagging)in which we take multiple random samples of instances from the training datacreate decision tree for each sampleand then predict the output for each instance by averaging the predictions of each of these trees by using randomization of samples and averaging results from multiple trees--an approach that is also known as the ensemble method--bagging will address some of the overfitting that results from single decision tree closer look at supervised algorithms