title: Limitations of sklearn Truncated SVD (LSA) Implementation
body: <p>I have the following scenario: I must analyze a big collection of text documents (around 3,000) and perform some clustering technic to gain some insight over it. To extract features I'm using tf-idf, but it yields more than 20,000 features which makes it difficult to achieve good results with clustering. So I started using LSA to reduce the dimensionality and improve the results. Here is a sample code using sci-kit learn's implementations:</p>
  
  <pre><code>dataset = fetch_20newsgroups(subset='all', categories=categories, 
                                shuffle=True, random_state=42)
  data = dataset.data
  labels = dataset.target
  
  vectorizer =TfidfVectorizer(use_idf=True, stop_words='english', ngram_range=(1,1), 
                               min_df=2, max_df=0.5, strip_accents='unicode',
                               smooth_idf=1, sublinear_tf=1)
  X = vectorizer.fit_transform(data)
  
  svd = TruncatedSVD(n_components=k)
  normalizer = Normalizer(copy=False)
  lsa = make_pipeline(svd, normalizer)
  
  X = lsa.fit_transform(X)
  
  # clustering over X
  </code></pre>
  
  <p>As I described above, I have more the 20,000 features and I wish I could reduce to half, for example, however if I use such big values for <code>n_components</code>, the <code>lsa.fit_transform(X)</code> returns exactly <code>n_components = n_documents</code>. The only values that work as expected are values bellow the number of documents. Is this something I'm missing from the theory or the implementation has this limitation? As far as I know from the theory of the LSA technic, the number of components to reduce the dimensionality to is unbounded and intuitively, I imagine that using a number of components that is closer to the number of features is better that use less than a fourth of it, because it would require less concepts to be grouped in more high level concepts. Please correct if I'm wrong.</p>