title: TRANSFORMERS: Asking to pad but the tokenizer does not have a padding token
body: <p>In trying to evaluate a bunch of transformers models sequentially with the same dataset to check which one performs better.</p>
  <p>The list of models is this one:</p>
  <pre><code>MODELS = [
        ('xlm-mlm-enfr-1024'   ,&quot;XLMModel&quot;),
        ('distilbert-base-cased', &quot;DistilBertModel&quot;),
        ('bert-base-uncased'     ,&quot;BertModel&quot;),
        ('roberta-base'        ,&quot;RobertaModel&quot;),
        (&quot;cardiffnlp/twitter-roberta-base-sentiment&quot;,&quot;RobertaSentTW&quot;),
        ('xlnet-base-cased'     ,&quot;XLNetModel&quot;),
        #('ctrl'                ,&quot;CTRLModel&quot;),
        ('transfo-xl-wt103'    ,&quot;TransfoXLModel&quot;),
        ('bert-base-cased'       ,&quot;BertModelUncased&quot;),
        ('xlm-roberta-base'     ,&quot;XLMRobertaModel&quot;),
        ('openai-gpt'           ,&quot;OpenAIGPTModel&quot;),
        ('gpt2'                 ,&quot;GPT2Model&quot;)
  </code></pre>
  <p>All of them work fine until 'ctrl' model, which returns this error:</p>
  <p><code>Asking to pad but the tokenizer does not have a padding token. Please select a token to use as 'pad_token' '(tokenizer.pad_token = tokenizer.eos_token e.g.)' or add a new pad token via 'tokenizer.add_special_tokens({'pad_token': '[PAD]'})'.</code></p>
  <p>When tokenizing the sentences of my dataset.</p>
  <p>The tokenizing code is</p>
  <pre><code>SEQ_LEN = MAX_LEN #(50)
  
  for pretrained_weights, model_name in MODELS:
  
  print(&quot;***************** INICIANDO &quot; ,model_name,&quot;, weights &quot;,pretrained_weights, &quot;********* &quot;)
  print(&quot;carganzo el tokenizador ()&quot;)
  tokenizer = AutoTokenizer.from_pretrained(pretrained_weights)
  print(&quot;creando el modelo preentrenado&quot;)
  transformer_model = TFAutoModel.from_pretrained(pretrained_weights)
  print(&quot;aplicando el tokenizador al dataset&quot;)
  
  ##APLICAMOS EL TOKENIZADOR##
  
  def tokenize(sentence):
    
    tokens = tokenizer.encode_plus(sentence, max_length=MAX_LEN,
                                 truncation=True, padding='max_length',
                                 add_special_tokens=True, return_attention_mask=True,
                                 return_token_type_ids=False, return_tensors='tf')
    return tokens['input_ids'], tokens['attention_mask']
  
  # initialize two arrays for input tensors
  Xids = np.zeros((len(df), SEQ_LEN))
  Xmask = np.zeros((len(df), SEQ_LEN))
  
  for i, sentence in enumerate(df['tweet']):
      Xids[i, :], Xmask[i, :] = tokenize(sentence)
      if i % 10000 == 0:
          print(i)  # do this so we can see some progress
  
  
  arr = df['label'].values  # take label column in df as array
  
  labels = np.zeros((arr.size, arr.max()+1))  # initialize empty (all zero) label array
  labels[np.arange(arr.size), arr] = 1  # add ones in indices where we have a value`
  </code></pre>
  <p>I have tried to define the padding tokens as the solution tells me, but then this error appears</p>
  <pre><code>could not broadcast input array from shape (3,) into shape (50,)
  </code></pre>
  <p>in line</p>
  <pre><code>Xids[i, :], Xmask[i, :] = tokenize(sentence)
  </code></pre>
  <p>I have also tried <a href="https://github.com/huggingface/transformers/issues/8452" rel="nofollow noreferrer">this solution</a> and doesn't work neither.</p>
  <p>If you have managed to read until here, thank you.</p>
  <p>Any help is needed.</p>