<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">I met the following error when I was using pytorch 1.6 with LSTM<br>
RuntimeError: cuDNN error: CUDNN_STATUS_BAD_PARAM</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">I think the root cause was that I was using the mixed precision training mode in pytorch 1.6 following <a href="https://pytorch.org/docs/stable/notes/amp_examples.html" rel="nofollow">https://pytorch.org/docs/stable/notes/amp_examples.html</a><br>
and the datatype might be converted to fp16 which causes this error.<br>
When I train my model with full precision, it worked normally.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/csarofeen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/csarofeen">@csarofeen</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ptrblck/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ptrblck">@ptrblck</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a></p>
<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">get cuDNN error: CUDNN_STATUS_BAD_PARAM in cnn-lstm network forward method</p>
<h2 dir="auto">To Reproduce</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import torch
from torch import nn, optim
from torch.cuda.amp import GradScaler, autocast

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(1, 10, 2)
        self.lstm = nn.LSTM(input_size=10, hidden_size=10,
                                batch_first=True,bidirectional=True)

    def forward(self, x):
        x = self.conv1(x)
        x = x.permute(0, 2, 1)
        x, _ = self.lstm(x)
        return x


net = Net().cuda()
net.train()
optimizer = optim.SGD(net.parameters(), lr=1e-3)

scaler = GradScaler()

optimizer.zero_grad()

with autocast():
    output = net(torch.rand(10,1,10).cuda())"><pre class="notranslate"><code class="notranslate">import torch
from torch import nn, optim
from torch.cuda.amp import GradScaler, autocast

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(1, 10, 2)
        self.lstm = nn.LSTM(input_size=10, hidden_size=10,
                                batch_first=True,bidirectional=True)

    def forward(self, x):
        x = self.conv1(x)
        x = x.permute(0, 2, 1)
        x, _ = self.lstm(x)
        return x


net = Net().cuda()
net.train()
optimizer = optim.SGD(net.parameters(), lr=1e-3)

scaler = GradScaler()

optimizer.zero_grad()

with autocast():
    output = net(torch.rand(10,1,10).cuda())
</code></pre></div>
<h2 dir="auto">Environment</h2>
<p dir="auto">torch==1.6.0 nightly (2020-04-10)</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mcarilli/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mcarilli">@mcarilli</a></p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/csarofeen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/csarofeen">@csarofeen</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ptrblck/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ptrblck">@ptrblck</a></p>