title: Python TfidfVectorizer throwing : empty vocabulary; perhaps the documents only contain stop words"
body: <p>I'm trying to use Python's Tfidf to transform a corpus of text. 
  However, when I try to fit_transform it, I get a value error ValueError: empty vocabulary; perhaps the documents only contain stop words. </p>
  
  <pre><code>In [69]: TfidfVectorizer().fit_transform(smallcorp)
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  &lt;ipython-input-69-ac16344f3129&gt; in &lt;module&gt;()
  ----&gt; 1 TfidfVectorizer().fit_transform(smallcorp)
  
  /Users/maxsong/anaconda/lib/python2.7/site-packages/sklearn/feature_extraction/text.pyc in fit_transform(self, raw_documents, y)
     1217         vectors : array, [n_samples, n_features]
     1218         """
  -&gt; 1219         X = super(TfidfVectorizer, self).fit_transform(raw_documents)
     1220         self._tfidf.fit(X)
     1221         # X is already a transformed view of raw_documents so
  
  /Users/maxsong/anaconda/lib/python2.7/site-packages/sklearn/feature_extraction/text.pyc in fit_transform(self, raw_documents, y)
      778         max_features = self.max_features
      779 
  --&gt; 780         vocabulary, X = self._count_vocab(raw_documents, self.fixed_vocabulary)
      781         X = X.tocsc()
      782 
  
  /Users/maxsong/anaconda/lib/python2.7/site-packages/sklearn/feature_extraction/text.pyc in _count_vocab(self, raw_documents, fixed_vocab)
      725             vocabulary = dict(vocabulary)
      726             if not vocabulary:
  --&gt; 727                 raise ValueError("empty vocabulary; perhaps the documents only"
      728                                  " contain stop words")
      729 
  
  ValueError: empty vocabulary; perhaps the documents only contain stop words
  </code></pre>
  
  <p>I read through the SO question here: <a href="https://stackoverflow.com/questions/14986893/problems-using-a-custom-vocabulary-for-tfidfvectorizer-scikit-learn">Problems using a custom vocabulary for TfidfVectorizer scikit-learn</a> and tried ogrisel's suggestion of using <strong>TfidfVectorizer(**params).build_analyzer()(dataset2) to check the results of the text analysis step</strong> and that seems to be working as expected: snippet below: </p>
  
  <pre><code>In [68]: TfidfVectorizer().build_analyzer()(smallcorp)
  Out[68]: 
  [u'due',
   u'to',
   u'lack',
   u'of',
   u'personal',
   u'biggest',
   u'education',
   u'and',
   u'husband',
   u'to',
  </code></pre>
  
  <p>Is there something else that I am doing wrong? the corpus I am feeding it is just one giant long string punctuated by newlines. </p>
  
  <p>Thanks!</p>