title: pytorch Rnn.py RuntimeError: CUDNN_STATUS_INTERNAL_ERROR
body: <p>I'm getting an CUDNN_STATUS_INTERNAL_ERROR error like below. </p>
  
  <p><code>python train_v2.py</code></p>
  
  <pre><code>Traceback (most recent call last):
    File "train_v2.py", line 113, in &lt;module&gt;
      main()
    File "train_v2.py", line 74, in main
      model.cuda()
    File "/home/ahkim/Desktop/squad_vteam/src/model.py", line 234, in cuda
      self.network.cuda()
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 249, in cuda
      return self._apply(lambda t: t.cuda(device))
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
      module._apply(fn)
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
      module._apply(fn)
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
      module._apply(fn)
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 112, in _apply
      self.flatten_parameters()
    File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 105, in flatten_parameters
      self.batch_first, bool(self.bidirectional))
  RuntimeError: CUDNN_STATUS_INTERNAL_ERROR
  </code></pre>
  
  <p>What should I try to resolve this issue?
  I tried deleting .nv but no success.</p>
  
  <hr>
  
  <p><code>nvidia-smi</code></p>
  
  <pre><code>Wed Aug  8 10:56:29 2018
  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 390.67                 Driver Version: 390.67                    |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  GeForce GTX TIT...  Off  | 00000000:04:00.0 Off |                  N/A |
  | 22%   21C    P8    15W / 250W |    125MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   1  GeForce GTX TIT...  Off  | 00000000:05:00.0 Off |                  N/A |
  | 22%   24C    P8    14W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   2  GeForce GTX TIT...  Off  | 00000000:08:00.0 Off |                  N/A |
  | 22%   23C    P8    14W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   3  GeForce GTX TIT...  Off  | 00000000:09:00.0 Off |                  N/A |
  | 22%   23C    P8    15W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   4  GeForce GTX TIT...  Off  | 00000000:85:00.0 Off |                  N/A |
  | 22%   24C    P8    14W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   5  GeForce GTX TIT...  Off  | 00000000:86:00.0 Off |                  N/A |
  | 22%   23C    P8    15W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   6  GeForce GTX TIT...  Off  | 00000000:89:00.0 Off |                  N/A |
  | 22%   21C    P8    15W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  |   7  GeForce GTX TIT...  Off  | 00000000:8A:00.0 Off |                  N/A |
  | 22%   23C    P8    15W / 250W |     11MiB / 12212MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  
  +-----------------------------------------------------------------------------+
  | Processes:                                                       GPU Memory |
  |  GPU       PID   Type   Process name                             Usage      |
  |=============================================================================|
  |    0      1603      C   /usr/bin/python                              114MiB |
  +-----------------------------------------------------------------------------+
  </code></pre>
  
  <hr>
  
  <h2>Update:</h2>
  
  <p>The same code runs without error using Nvidia 
  <code>Driver Version: 396.26</code> (cuda V9.1.85. torch.backends.cudnn.version():
  7102). I'm getting an error using <code>Driver Version: 390.67</code> (cuda V9.1.85. torch.backends.cudnn.version(): 7102)</p>