title: RuntimeError: CUDA error: device-side assert triggered - BART model
body: <p>I am trying to run BART language model for a text generation task.</p>
  <p>My code was working fine when I used for another encoder-decoder model (T5), but with bart I am getting this error:</p>
  <pre><code>File &quot;train_bart.py&quot;, line 89, in train
      outputs = model(input_ids = ids, attention_mask = mask, decoder_input_ids=y_ids, labels=lm_labels)                                                     cs-lab-host1&quot; 12:39 10-Aug-21
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/torch/nn/modules/module.py&quot;, line 1051, in _call_impl
      return forward_call(*input, **kwargs)
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/transformers/models/bart/modeling_bart.py&quot;, line 1308, in forward
      return_dict=return_dict,
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/torch/nn/modules/module.py&quot;, line 1051, in _call_impl
      return forward_call(*input, **kwargs)
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/transformers/models/bart/modeling_bart.py&quot;, line 1196, in forward
      return_dict=return_dict,
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/torch/nn/modules/module.py&quot;, line 1051, in _call_impl
      return forward_call(*input, **kwargs)
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/transformers/models/bart/modeling_bart.py&quot;, line 985, in forward
      attention_mask, input_shape, inputs_embeds, past_key_values_length
    File &quot;.../venv/tf_23/lib/python3.6/site-packages/transformers/models/bart/modeling_bart.py&quot;, line 866, in _prepare_decoder_attent
  ion_mask
      ).to(self.device)
  RuntimeError: CUDA error: device-side assert triggered
  CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
  For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
  </code></pre>
  <p>And this is where error happens:</p>
  <pre><code>for _, data in tqdm(enumerate(loader, 0), total=len(loader), desc='Processing batches..'):
      y = data['target_ids'].to(device, dtype = torch.long)
      y_ids = y[:, :-1].contiguous()
      lm_labels = y[:, 1:].clone().detach()
      lm_labels[y[:, 1:] == tokenizer.pad_token_id] = -100
      ids = data['source_ids'].to(device, dtype = torch.long)
      mask = data['source_mask'].to(device, dtype = torch.long)
  
      outputs = model(input_ids = ids, attention_mask = mask, decoder_input_ids=y_ids, labels=lm_labels)
      loss = outputs[0]
  </code></pre>
  <p><code>loader</code> is the tokenized and processed data.</p>