title: AttributeError: lower not found; using a Pipeline with a CountVectorizer in scikit-learn
body: <p>I have a corpus as such:</p>
  
  <pre><code>X_train = [ ['this is an dummy example'] 
        ['in reality this line is very long']
        ...
        ['here is a last text in the training set']
      ]
  </code></pre>
  
  <p>and some labels:</p>
  
  <pre><code>y_train = [1, 5, ... , 3]
  </code></pre>
  
  <p>I would like to use Pipeline and GridSearch as follows:</p>
  
  <pre><code>pipeline = Pipeline([
      ('vect', CountVectorizer()),
      ('tfidf', TfidfTransformer()),
      ('reg', SGDRegressor())
  ])
  
  
  parameters = {
      'vect__max_df': (0.5, 0.75, 1.0),
      'tfidf__use_idf': (True, False),
      'reg__alpha': (0.00001, 0.000001),
  }
  
  grid_search = GridSearchCV(pipeline, parameters, n_jobs=1, verbose=1)
  
  grid_search.fit(X_train, y_train)
  </code></pre>
  
  <p>When I run this, I get an error saying <code>AttributeError: lower not found</code>.  </p>
  
  <p>I searched and found a question about this error <a href="https://stackoverflow.com/questions/28103992/tfidf-vectorizer-giving-error">here</a>, which lead me to believe that there was a problem with my text not being tokenized (which sounded like it hit the nail on the head, since I was using a list of list as input data, where each list contained one single unbroken string). </p>
  
  <p>I cooked up a quick and dirty tokenizer to test this theory:</p>
  
  <pre><code>def my_tokenizer(X):
      newlist = []
      for alist in X:
          newlist.append(alist[0].split(' '))
      return newlist
  </code></pre>
  
  <p>which does what it is supposed to, but when I use it in the arguments to the <code>CountVectorizer</code>:  </p>
  
  <pre><code>pipeline = Pipeline([
      ('vect', CountVectorizer(tokenizer=my_tokenizer)),
  </code></pre>
  
  <p>...I still get the same error as if nothing happened.</p>
  
  <p>I did notice that I can circumvent the error by commenting out the <code>CountVectorizer</code> in my Pipeline. Which is strange...I didn't think you could use the <code>TfidfTransformer()</code> without first having a data structure to transform...in this case the matrix of counts.  </p>
  
  <p>Why do I keep getting this error? Actually, it would be nice to know what this error means! (Was <code>lower</code> called to convert the text to lowercase or something? I can't tell from reading the stack trace). Am I misusing the Pipeline...or is the problem really an issue with the arguments to the <code>CountVectorizer</code> alone?  </p>
  
  <p>Any advice would be greatly appreciated.</p>