title: CountVectorizer() : AttributeError: 'numpy.float64' object has no attribute 'lower'
body: <p>I am trying to fit a dataset which has event_type and notes (free text) columns. before calling the MultinomialNB model , I processed the text and converted it to array to vectorize it and calculate the tfidf here below the code provided:</p>
  
  <h1>Convert Event types from string to integer for easy processing</h1>
  
  <pre><code>ACLED['category_id'] = ACLED['event_type'].factorize()[0]
  category_id_ACLED = ACLED[['event_type', 'category_id']].drop_d
  
  uplicates().sort_values('category_id')
  category_to_id = dict(category_id_ACLED.values)
  id_to_category = dict(category_id_ACLED[['category_id', 'event_type']].values)
  </code></pre>
  
  <h1>Text Representation</h1>
  
  <p>I also converted notes and category_id into features and labels as follows:</p>
  
  <pre><code>tfidf = TfidfVectorizer(sublinear_tf=True, min_df=5, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')
  features = tfidf.fit_transform(ACLED.notes).toarray()
  labels = ACLED.category_id
  print(features.shape)
  </code></pre>
  
  <p>Then I split the dataset to training and testing sets using features and labels :</p>
  
  <pre><code>X_train, X_test, y_train, y_test = train_test_split(features ,labels, random_state=0)
  print('Original dataset shape {}'.format(Counter(y_train)))
  </code></pre>
  
  <h1>output</h1>
  
  <pre><code>Original dataset shape Counter({1: 1280, 2: 819, 0: 676, 3: 593, 4: 138, 5: 53, 7: 50, 6: 21, 8: 10})
  </code></pre>
  
  <p>Since the classes are imbalanced , I used SMOTE to resolve the minority issue and creating synthethic copies </p>
  
  <h1>Apply the random over-sampling to overcome imbalanced classes</h1>
  
  <pre><code>sm = SMOTE(random_state=42)
  X_resampled, y_resampled = sm.fit_sample(X_train, y_train)
  print('Resampled dataset shape {}'.format(Counter(y_resampled)))
  </code></pre>
  
  <h1>output after oversampling</h1>
  
  <pre><code>Resampled dataset shape Counter({3: 1280, 1: 1280, 2: 1280, 0: 1280, 7: 1280, 6: 1280, 4: 1280, 5: 1280, 8: 1280})
  </code></pre>
  
  <p>Everything until now is working fine , until I tried to calculate the terms frequency using CountVectorizer() as follows :</p>
  
  <pre><code>count_vect = CountVectorizer()
  X_train_counts = count_vect.fit_transform(X_resampled)
  tfidf_transformer = TfidfTransformer()
  X_train_tfidf = tfidf_transformer.fit_transform(X_train_counts)
  </code></pre>
  
  <h1>Output error :</h1>
  
  <pre><code>'numpy.ndarray' object has no attribute 'lower'
  </code></pre>
  
  <p>I tried to use ravel() function to flatten the array but the error persists, any ideas, thanks in advance</p>