title: Pytorch: RuntimeError: reduce failed to synchronize: cudaErrorAssert: device-side assert triggered
body: <p>I am running into the following error when trying to train <a href="https://github.com/natanielruiz/deep-head-pose/blob/master/code/test_hopenet.py" rel="noreferrer">this</a> on <a href="https://drive.google.com/file/d/0B7OEHD3T4eCkVGs0TkhUWFN6N1k/view" rel="noreferrer">this dataset</a>.</p>
  
  <p>Since this is the configuration published in the paper, I am assuming I am doing something incredibly wrong.</p>
  
  <p>This error arrives on a different image every time I try to run training.</p>
  
  <pre><code>C:/w/1/s/windows/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: block: [0,0,0], thread: [6,0,0] Assertion `t &gt;= 0 &amp;&amp; t &lt; n_classes` failed.
  Traceback (most recent call last):
    File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.1\helpers\pydev\pydevd.py", line 1741, in &lt;module&gt;
      main()
    File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.1\helpers\pydev\pydevd.py", line 1735, in main
      globals = debugger.run(setup['file'], None, None, is_module)
    File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.1\helpers\pydev\pydevd.py", line 1135, in run
      pydev_imports.execfile(file, globals, locals)  # execute the script
    File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.1\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
      exec(compile(contents+"\n", file, 'exec'), glob, loc)
    File "C:/Noam/Code/vision_course/hopenet/deep-head-pose/code/original_code_augmented/train_hopenet_with_validation_holdout.py", line 187, in &lt;module&gt;
      loss_reg_yaw = reg_criterion(yaw_predicted, label_yaw_cont)
    File "C:\Noam\Code\vision_course\hopenet\venv\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
      result = self.forward(*input, **kwargs)
    File "C:\Noam\Code\vision_course\hopenet\venv\lib\site-packages\torch\nn\modules\loss.py", line 431, in forward
      return F.mse_loss(input, target, reduction=self.reduction)
    File "C:\Noam\Code\vision_course\hopenet\venv\lib\site-packages\torch\nn\functional.py", line 2204, in mse_loss
      ret = torch._C._nn.mse_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
  RuntimeError: reduce failed to synchronize: cudaErrorAssert: device-side assert triggered
  </code></pre>
  
  <hr>
  
  <p>Any ideas?</p>