title: keras - AttributeError: 'numpy.ndarray' object has no attribute 'lower'
body: <p>Not sure what am I doing wrong:</p>
  <p>I've a dataset (all text) and when I try to fit the Tokenizer it fails.</p>
  <pre><code>from keras.preprocessing.text import Tokenizer as Tok
  
  # The maximum number of words to be used. (most frequent)
  MAX_NB_WORDS = 50000
  # Max number of words in each complaint.
  MAX_SEQUENCE_LENGTH = 250
  # This is fixed.
  EMBEDDING_DIM = 100
  
  labels = training_data['group_name']
  features = training_data.drop('group_name', axis='columns')
  
  tokenizer = Tok(num_words=MAX_NB_WORDS, filters='!&quot;#$%&amp;()*+,-./:;&lt;=&gt;?@[\]^_`{|}~', lower=True)
  tokenizer.fit_on_texts(features.values)
  </code></pre>
  <p><code>AttributeError: 'numpy.ndarray' object has no attribute 'lower'</code></p>
  <p>I've cleaned the text as follows:</p>
  <pre><code>REPLACE_BY_SPACE_RE = re.compile('[/(){}\[\]\|@,;]')
  BAD_SYMBOLS_RE = re.compile('[^0-9a-z #+_]')
  STOPWORDS = set(stopwords.words('english'))
  
  
  def clean_text(text):   
      text = text.lower() # lowercase text
      text = REPLACE_BY_SPACE_RE.sub(' ', text) # replace REPLACE_BY_SPACE_RE symbols by space in text. substitute the matched string in REPLACE_BY_SPACE_RE with space.
      text = BAD_SYMBOLS_RE.sub('', text) # remove symbols which are in BAD_SYMBOLS_RE from text. substitute the matched string in BAD_SYMBOLS_RE with nothing. 
      text = text.replace('x', '')
  
      text = ' '.join(word for word in text.split() if word not in STOPWORDS) # remove stopwors from text
      return text
  
  print('clean text')
  training_data = training_data.applymap(lambda x: clean_text(x))
  </code></pre>
  <p>... so I can't see where the <code>numpy.ndarray</code> comes from</p>
  <p><strong>UPDATE:</strong></p>
  <p>I was able to understand the problem and workaround it in an ugly manner:</p>
  <p>Consolidate all columns to a single one:</p>
  <pre><code>labels = df['group_name']
  features = df.drop('group_name', axis='columns')
  tmp = pd.DataFrame()
  tmp['txt'] = features[features.columns[1:]].apply(
      lambda x: ','.join(x.dropna().astype(str)),
      axis=1
  )
  </code></pre>
  <p>now it passes the problematic step:</p>
  <pre><code>tokenizer = Tok(num_words=MAX_NB_WORDS, filters='!&quot;#$%&amp;()*+,-./:;&lt;=&gt;?@[\]^_`{|}~', lower=True)
  tokenizer.fit_on_texts(tmp['txt'].values)
  word_index = tokenizer.word_index
  print('Found %s unique tokens.' % len(word_index))
  X = tokenizer.texts_to_sequences(tmp['txt'].values)
  X = pad_sequences(X, maxlen=MAX_SEQUENCE_LENGTH)
  print('Shape of data tensor:', X.shape)
  Y = pd.get_dummies(labels).values
  print('Shape of label tensor:', Y.shape)
  </code></pre>
  <p>However I still want to keep the original columns and don't keep all data in a single column.</p>
  <p>How can I achieve that (i.e. get all values from all columns without iterate on nested numpy arrays)?</p>