title: Loss is “nan” when fine-tuning HuggingFace NLI model (both RoBERTa/BART)
body: <p>I'm using <a href="https://huggingface.co/" rel="nofollow noreferrer">HuggingFace</a>'s Transformer's library and I’m trying to fine-tune a pre-trained NLI model (<code>ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli</code>) on a dataset of around 276.000 hypothesis-premise pairs. I’m following the instructions from the docs <a href="https://huggingface.co/transformers/custom_datasets.html" rel="nofollow noreferrer">here</a> and <a href="https://huggingface.co/transformers/training.html" rel="nofollow noreferrer">here</a>. I have the impression that the fine-tuning works (it does the training and saves the checkpoints), but <code>trainer.train()</code> and <code>trainer.evaluate()</code> return &quot;nan&quot; for the loss.</p>
  <p><strong>What I've tried:</strong></p>
  <ul>
  <li>I tried using both <code>ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli</code> and <code>facebook/bart-large-mnli</code> to make sure that it's not linked to specific model, but I get the issue for both models</li>
  <li>I tried following the advice in this <a href="https://github.com/huggingface/transformers/issues/1727" rel="nofollow noreferrer">related github issue</a>, but adding <code>num_labels=3</code> to the config file does not solve the issue. (I think my issue is different because the models are already fine-tuned on NLI in my case)</li>
  <li>I tried many different ways of changing my input data because I suspected that there could be an issue with my input data, but I also couldn't solve it that way.</li>
  <li><strong>The probable source of the issue:</strong> <em>I inspected the prediction output from the model during training and the weird thing is the prediction value always seems to be &quot;0&quot; (entailment) in 100% of cases (see printed output in the code below). This is clearly an error. I think the source for this is that the logits the model seems to return during training are <code>torch.tensor([[np.nan, np.nan, np.nan]])</code> and when you apply <code>.argmax(-1)</code> to this, you get torch.tensor(0). The big mystery for me is why the logits would become &quot;nan&quot;, because the model does not do that when I use the same input data only outside of the trainer.
  =&gt; Does anyone know where this issues comes from? See my code below.</em></li>
  </ul>
  <p>Thanks a lot in advance for any suggestion!</p>
  <h3>Here is my code:</h3>
  <pre><code>### load model &amp; tokenize
  from transformers import AutoTokenizer, AutoModelForSequenceClassification
  import torch
  
  max_length = 256
  hg_model_hub_name = &quot;ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli&quot;
  # also tried: hg_model_hub_name = &quot;facebook/bart-large-mnli&quot;
  tokenizer = AutoTokenizer.from_pretrained(hg_model_hub_name)
  model = AutoModelForSequenceClassification.from_pretrained(hg_model_hub_name)
  model.config
  
  device = &quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;
  print(f&quot;Device: {device}&quot;)
  if device == &quot;cuda&quot;:
    model = model.half()
  model.to(device)
  model.train();
  
  #... some data preprocessing
  
  encodings_train = tokenizer(premise_train, hypothesis_train, return_tensors=&quot;pt&quot;, max_length=max_length,
                              return_token_type_ids=True, truncation=False, padding=True)
  encodings_val = tokenizer(premise_val, hypothesis_val, return_tensors=&quot;pt&quot;, max_length=max_length,
                            return_token_type_ids=True, truncation=False, padding=True)
  encodings_test = tokenizer(premise_test, hypothesis_test, return_tensors=&quot;pt&quot;, max_length=max_length,
                             return_token_type_ids=True, truncation=False, padding=True)
  
  
  ### create pytorch dataset object
  class XDataset(torch.utils.data.Dataset):
      def __init__(self, encodings, labels):
          self.encodings = encodings
          self.labels = labels
      def __getitem__(self, idx):
          item = {key: torch.as_tensor(val[idx]) for key, val in self.encodings.items()}
          #item = {key: torch.as_tensor(val[idx]).to(device) for key, val in self.encodings.items()}
          item['labels'] = torch.as_tensor(self.labels[idx])
          #item['labels'] = self.labels[idx]
          return item
      def __len__(self):
          return len(self.labels)
  
  dataset_train = XDataset(encodings_train, label_train)
  dataset_val = XDataset(encodings_val, label_val)
  dataset_test = XDataset(encodings_test, label_test)
  
  # compute metrics with trainer
  from sklearn.metrics import accuracy_score, precision_recall_fscore_support
  def compute_metrics(pred):
      labels = pred.label_ids
      print(labels)
      preds = pred.predictions.argmax(-1)
      print(preds)
      precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='binary', pos_label=0)
      acc = accuracy_score(labels, preds)
      return {
          'accuracy': acc,
          'f1': f1,
          'precision': precision,
          'recall': recall
      }
  
  
  ## training
  from transformers import Trainer, TrainingArguments
  
  # https://huggingface.co/transformers/main_classes/trainer.html#transformers.TrainingArguments
  training_args = TrainingArguments(
      output_dir='./results',          # output directory
      num_train_epochs=1,              # total number of training epochs
      per_device_train_batch_size=8,  # batch size per device during training
      per_device_eval_batch_size=8,   # batch size for evaluation
      warmup_steps=500,                # number of warmup steps for learning rate scheduler
      weight_decay=0.01,               # strength of weight decay
      logging_dir='./logs',            # directory for storing logs
      logging_steps=100,
  )
  
  trainer = Trainer(
      model=model,                         # the instantiated  Transformers model to be trained
      args=training_args,                  # training arguments, defined above
      train_dataset=dataset_train,         # training dataset
      eval_dataset=dataset_val             # evaluation dataset
  )
  
  trainer.train()
  # output: TrainOutput(global_step=181, training_loss=nan)
  trainer.evaluate()
  # output: 
  [2 2 2 0 0 2 2 2 0 2 0 0 2 2 2 2 0 2 0 2 2 2 2 0 2 0 2 0 0 2 0 0 2 0 0 0 2
   0 2 0 0 0 0 0 2 0 0 2 2 2 0 2 2 2 2 2 0 0 0 0 2 0 0 0 2 2 0 0 0 2 0 0 0 2
   2 0 2 0 0 2 2 2 0 2 2 0 0 0 0 0 0 0 2 0 0 0 0 2 0 2 2 0 2 0 0 2 2 2 2 2 2
   2 0 0 0 0 2 0 0 2 0 0 0 0 2 2 2 0 0 0 0 0 2 0 0 2 0 2 0 2 0 2 0 0 2 2 0 0
   2 2 2 2 2 2 0 0 2 2 2 2 0 2 0 0 2 2 2 0 0 2 0 2 0 2 0 0 0 0 0 0 2 0 0 2 2
   0 2 2 2 0 2 2 0 2 2 2 2 2 2 0 0 2 0 0 2 2 0 0 0 2 0 2 2 2 0 0 0 0 0 0 0 0
   2 0 2 2 2 0 2 0 0 2 0 2 2 0 0 0 0 2 2 2 0 0 0 2 2 2 2 0 2 0 2 2 2]
  [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
  
  {'epoch': 1.0,
   'eval_accuracy': 0.5137254901960784,
   'eval_f1': 0.6787564766839378,
   'eval_loss': nan,
   'eval_precision': 0.5137254901960784,
   'eval_recall': 1.0}
  </code></pre>
  <p><strong>Edit:</strong>  I've also opened a github issue with a but more detailed description of the issue here: <a href="https://github.com/huggingface/transformers/issues/9160" rel="nofollow noreferrer">https://github.com/huggingface/transformers/issues/9160</a></p>