feature extraction with principal component analysis principal component analysispopularly known as pcais statistical method that uses the process of linearorthogonal transformation to transform higher-dimensional set of features that could be possibly correlated into lower-dimensional set of linearly uncorrelated features these transformed and newly created features are also known as principal components or pcs in any pca transformationthe total number of pcs is always less than or equal to the initial number of features the first principal component tries to capture the maximum variance of the original set of features each of the succeeding components tries to capture more of the variance such that they are orthogonal to the preceding components an important point to remember is that pca is sensitive to feature scaling our main task is to take set of initial features with dimension let' say and reduce it to subset of extracted principal components of lower dimension ld the matrix decomposition process of singular value decomposition is extremely useful in helping us obtain the principal components you can quickly refresh your memory on svd by referring to the sub-section of "singular value decompositionunder the "important conceptsin the "mathematicssection in to check out the necessary mathematical formula and concepts considering we have data matrix ( )where we have observations and dimensions (features)we can depict svd of the feature matrix as ( ( )usvt such that all the principal components are contained in the component vtwhich can be depicted as followse pc ( ' pc ( ' ' / pc ' ) the principal components are represented by {pc pc pcdwhich are all one-dimensional vectors of dimensions ( dfor extracting the first principal componentswe can first transpose this matrix to obtain the following representation pc ' ( pc pc / pc ee uu now we can extract out the first principal components such that < and the reduced principal component set can be depicted as follows pc ' ( pc pc / pc ee uu finallyto perform dimensionality reductionwe can get the reduced feature set using the following mathematical transformation ( df( )pc( dwhere the dot product between the original feature matrix and the reduced subset of principal components gives us reduced feature set of features very important point to remember here is that you might need to center your initial feature matrix by removing the mean because by defaultpca assumes that your data is centered around the origin