sketching our roadmap dimensionality reduction can be roughly grouped into feature selection and feature extraction methods we have already employed some kind of feature selection in almost every when we inventedanalyzedand then probably dropped some features in this we will present some ways that use statistical methodsnamely correlation and mutual informationto be able to do feature selection in vast feature spaces feature extraction tries to transform the original feature space into lower-dimensional feature space this is useful especially when we cannot get rid of features using selection methodsbut we still have too many features for our learner we will demonstrate this using principal component analysis (pca)linear discriminant analysis (lda)and multidimensional scaling (mdsselecting features if we want to be nice to our machine learning algorithmwe will provide it with features that are not dependent on each otheryet highly dependent on the value to be predicted it means that each feature adds some salient information removing any of the features will lead to drop in performance if we have only handful of featureswe could draw matrix of scatter plots one scatter plot for every feature-pair combination relationships between the features could then be easily spotted for every feature pair showing an obvious dependencewe would then think whether we should remove one of them or better design newercleaner feature out of both most of the timehoweverwe have more than handful of features to choose from just think of the classification task where we had bag-of-words to classify the quality of an answerwhich would require , by , scatter plot in this casewe need more automated way to detect the overlapping features and way to resolve them we will present two general ways to do so in the following subsectionsnamely filters and wrappers