title: How to prepare text for BERT - getting error
body: <p>I am trying to learn BERT for text classification. I am finding some problem in preparing data for using BERT.</p>
  <p>From my Dataset, I am segregating the sentiments and reviews as:</p>
  <pre><code>X = df['sentiments']
  y = df['reviews'] #it contains four different class of reviews
  </code></pre>
  <p>Next,</p>
  <pre><code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
  train_encodings = tokenizer(X_train, truncation=True, padding=True, max_length=512) 
  </code></pre>
  <p>Here is where I get error:</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-js lang-js prettyprint-override"><code>ValueError                                Traceback (most recent call last)
  &lt;ipython-input-70-22714fcf7991&gt; in &lt;module&gt;()
  ----&gt; 1 train_encodings = tokenizer(X_train, truncation=True, padding=True, max_length=max_length)
        2 #valid_encodings = tokenizer(valid_texts, truncation=True, padding=True, max_length=max_length)
  
  /usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in __call__(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
     2261         if not _is_valid_text_input(text):
     2262             raise ValueError(
  -&gt; 2263                 "text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) "
     2264                 "or `List[List[str]]` (batch of pretokenized examples)."
     2265             )
  
  ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples).</code></pre>
  </div>
  </div>
  </p>
  <p>When I am trying to convert X to list and use it, I get another error:</p>
  <pre><code>TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]
  </code></pre>
  <p>Can someone please explain where the problem is? Previously I followed a tutorial on 20 news dataset and it worked. But now when I am using it for another project, it doesn't work and I feel sad.</p>
  <p>Thanks.</p>