title: Dimensionality reduction using PCA for text classification
body: <p>I am doing text classification of documents, I have around 4k categories and 1.1 million data samples. </p>
  
  <p>I am constructing matrix which contain frequency of words in each document.
  The sample of matrix looks as below</p>
  
  <pre><code>            X1     X2     X3        X4 
  D1          1      1       0        1
  D2          1      1       1        0
  D3          1      1       0        0
  D4          1      1       1        1
  D5          0      0       1        0
  D6          0      0       1        1
  </code></pre>
  
  <p>In above matrix, X1 and X2 are redundant features because they have same values in all rows.</p>
  
  <p>First when I construct matrix from 1.1 million data, I will get huge matrix with 90k features.</p>
  
  <p>To reduce matrix dimension, I am using dimension reduction technique PCA
   I have used TruncatedSVD to calculate PCA as I am using sparse matrix.</p>
  
  <p>I am using Sckit learn implementation of PCA using below code</p>
  
  <pre><code>from sklearn.decomposition import TruncatedSVD
  X = [[1,1,0,1], [1,1,1,0], [1,1,0,0],[1,1,1,1],[0,0,1,0],[0,0,1,1]]
  svd = TruncatedSVD(n_components=3)
  svd.fit(X)  
  X_new=svd.fit_transform(X)
  </code></pre>
  
  <p>The output of X_new is </p>
  
  <pre><code>array([[ 1.53489494, -0.49612748, -0.63083679],
     [ 1.57928583, -0.04762643,  0.70963934],
     [ 1.13759356, -0.80736818,  0.2324597 ],
     [ 1.97658721,  0.26361427, -0.15365716],
     [ 0.44169227,  0.75974175,  0.47717963],
     [ 0.83899365,  1.07098246, -0.38611686]])
  </code></pre>
  
  <p>This is the reduced dimension I got
  I am giving X_new as input to Naive Bayes classifier.</p>
  
  <pre><code>clf = GaussianNB()
  model=clf.fit(X_new, Y)
  </code></pre>
  
  <p>For 1.1 million sample I got below outputs:</p>
  
  <pre><code>No_of_components
  (“n_components” parameter)           accuracy
     1000                                6.57%
     500                                 7.25%
     100                                 5.72%
  </code></pre>
  
  <p>I am getting very low accuracy,</p>
  
  <p>Whether  above steps are correct?</p>
  
  <p>What are the things I need to include further?</p>