feature extraction at some pointafter we have removed the redundant features and dropped the irrelevant oneswe often still find that we have too many features no matter what learning method we usethey all perform badlyand given the huge feature spacewe understand that they actually cannot do better we realize that we have to cut living flesh and that we have to get rid of features that all common sense tells us are valuable another situation when we need to reduce the dimensionsand when feature selection does not help muchis when we want to visualize data thenwe need to have at most three dimensions at the end to provide any meaningful graph enter the feature extraction methods they restructure the feature space to make it more accessible to the modelor simply cut down the dimensions to two or three so that we can show dependencies visually againwe can distinguish between feature extraction methods as being linear or non-linear ones and as beforein the feature selection sectionwe will present one method for each typeprincipal component analysis for linear and multidimensional scaling for the non-linear version although they are widely known and usedthey are only representatives for many more interesting and powerful feature extraction methods about principal component analysis (pcaprincipal component analysis is often the first thing to try out if you want to cut down the number of features and do not know what feature extraction method to use pca is limited as it is linear methodbut chances are that it already goes far enough for your model to learn well enough add to that the strong mathematical properties it offersthe speed at which it finds the transformed feature spaceand its ability to transform between the original and transformed features laterwe can almost guarantee that it will also become one of your frequently used machine learning tools summarizing itgiven the original feature spacepca finds linear projection of it into lower dimensional space that has the following propertiesthe conserved variance is maximized the final reconstruction error (when trying to go back from transformed features to original onesis minimized as pca simply transforms the input datait can be applied both to classification and regression problems in this sectionwe will use classification task to discuss the method