title: Mapping text data through huggingface tokenizer
body: <p>I have my encode function that looks like this:</p>
  
  <pre class="lang-py prettyprint-override"><code>from transformers import BertTokenizer, BertModel
  
  MODEL = 'bert-base-multilingual-uncased'
  tokenizer = BertTokenizer.from_pretrained(MODEL)
  
  def encode(texts, tokenizer=tokenizer, maxlen=10):
  #     import pdb; pdb.set_trace()
      inputs = tokenizer.encode_plus(
          texts,
          return_tensors='tf',
          return_attention_masks=True, 
          return_token_type_ids=True,
          pad_to_max_length=True,
          max_length=maxlen
      )
  
      return inputs['input_ids'], inputs["token_type_ids"], inputs["attention_mask"]
  </code></pre>
  
  <p>I want to get my data encoded on the fly by doing this:</p>
  
  <pre class="lang-py prettyprint-override"><code>x_train = (tf.data.Dataset.from_tensor_slices(df_train.comment_text.astype(str).values)
             .map(encode))
  </code></pre>
  
  <p>However, this chucks the error:</p>
  
  <pre><code>ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers.
  </code></pre>
  
  <p>Now from my understanding when I set a breakpoint inside <code>encode</code> it was because I was sending a non-numpy array. How do I get huggingface transformers to play nice with tensorflow strings as inputs?</p>
  
  <p>If you need a dummy dataframe here it is:</p>
  
  <pre><code>df_train = pd.DataFrame({'comment_text': ['Today was a good day']*5})
  </code></pre>
  
  <h2>What I tried</h2>
  
  <p>So I tried to use <code>from_generator</code> so that I can parse in the strings to the <code>encode_plus</code> function. However, this does not work with TPUs.</p>
  
  <pre class="lang-py prettyprint-override"><code>AUTO = tf.data.experimental.AUTOTUNE
  
  def get_gen(df):
      def gen():
          for i in range(len(df)):
              yield encode(df.loc[i, 'comment_text']) , df.loc[i, 'toxic']
      return gen
  
  shapes = ((tf.TensorShape([maxlen]), tf.TensorShape([maxlen]), tf.TensorShape([maxlen])), tf.TensorShape([]))
  
  train_dataset = tf.data.Dataset.from_generator(
      get_gen(df_train),
      ((tf.int32, tf.int32, tf.int32), tf.int32),
      shapes
  )
  train_dataset = train_dataset.batch(BATCH_SIZE).prefetch(AUTO)
  </code></pre>
  
  <h2>Version Info:</h2>
  
  <p><code>transformers.__version__, tf.__version__</code> => <code>('2.7.0', '2.1.0')</code></p>