title: raise ValueError("np.nan is an invalid document, expected byte or "
body: <p>I am using <code>CountVectorizer()</code> in scikit-learn for vectorizing the feature sequence. I am receiving an error as below:</p>
  <pre><code>ValueError: np.nan is an invalid document, expected byte or unicode string.
  </code></pre>
  <p>I am using an example csv dataset with two columns <code>CONTENT</code> and <code>sentiment</code>.</p>
  <p>Here's my code:</p>
  <pre><code>df = pd.read_csv(&quot;train.csv&quot;,encoding = &quot;ISO-8859-1&quot;)
  X, y = df.CONTENT, df.sentiment
  
  X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
  print X_train, y_train
  
  vect = CountVectorizer(ngram_range=(1,3), analyzer='word', encoding = &quot;ISO-8859-1&quot;)
  print vect
  X=vect.fit_transform(X_train, y_train)
  y=vect.fit(X_test) 
  print vect.get_feature_names()
  </code></pre>
  <p>Here is the error message in full:</p>
  <pre><code>File &quot;C:/Users/HP/cntVect.py&quot;, line 28, in &lt;module&gt;
      X=vect.fit_transform(X_train, y_train)
  
    File &quot;C:\ProgramData\Anaconda2\lib\site-packages\sklearn\feature_extraction\text.py&quot;, line 839, in fit_transform
      self.fixed_vocabulary_)
  
    File &quot;C:\ProgramData\Anaconda2\lib\site-packages\sklearn\feature_extraction\text.py&quot;, line 762, in _count_vocab
      for feature in analyze(doc):
  
    File &quot;C:\ProgramData\Anaconda2\lib\site-packages\sklearn\feature_extraction\text.py&quot;, line 241, in &lt;lambda&gt;
      tokenize(preprocess(self.decode(doc))), stop_words)
  
    File &quot;C:\ProgramData\Anaconda2\lib\site-packages\sklearn\feature_extraction\text.py&quot;, line 121, in decode
      raise ValueError(&quot;np.nan is an invalid document, expected byte or &quot;
  
  ValueError: np.nan is an invalid document, expected byte or unicode string.
  </code></pre>