<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">It is advertised, that forward pass of JITed RNNs (e.g. GRU) is as fast as cuDNN implementation.<br>
But it is not the case.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<p dir="auto">See here: <a href="https://gist.github.com/usamec/af21be7b83e6b1a3f38c26136af811f3">https://gist.github.com/usamec/af21be7b83e6b1a3f38c26136af811f3</a></p>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Forward pass is as fast as cuDNN.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: GPU 0: TITAN Xp
Nvidia driver version: 430.50
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.0

Versions of relevant libraries:
[pip3] numpy==1.15.0
[conda] _pytorch_select           0.2                       gpu_0  
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.0.130                      0  
[conda] mkl                       2019.4                      243  
[conda] mkl-service               2.3.0            py36he904b0f_0  
[conda] mkl_fft                   1.0.15           py36ha843d7b_0  
[conda] mkl_random                1.1.0            py36hd6b4f25_0  
[conda] numpy                     1.18.1           py36h4f9e942_0  
[conda] numpy-base                1.18.1           py36hde5b4d6_1  
[conda] pytorch                   1.4.0           py3.6_cuda10.0.130_cudnn7.6.3_0    pytorch
[conda] pytorch-qrnn              0.2.1                    pypi_0    pypi
[conda] torch-scatter             1.4.0                    pypi_0    pypi
[conda] torchvision               0.5.0                    pypi_0    pypi"><pre class="notranslate"><code class="notranslate">Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: GPU 0: TITAN Xp
Nvidia driver version: 430.50
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.0

Versions of relevant libraries:
[pip3] numpy==1.15.0
[conda] _pytorch_select           0.2                       gpu_0  
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.0.130                      0  
[conda] mkl                       2019.4                      243  
[conda] mkl-service               2.3.0            py36he904b0f_0  
[conda] mkl_fft                   1.0.15           py36ha843d7b_0  
[conda] mkl_random                1.1.0            py36hd6b4f25_0  
[conda] numpy                     1.18.1           py36h4f9e942_0  
[conda] numpy-base                1.18.1           py36hde5b4d6_1  
[conda] pytorch                   1.4.0           py3.6_cuda10.0.130_cudnn7.6.3_0    pytorch
[conda] pytorch-qrnn              0.2.1                    pypi_0    pypi
[conda] torch-scatter             1.4.0                    pypi_0    pypi
[conda] torchvision               0.5.0                    pypi_0    pypi
</code></pre></div>
<p dir="auto">I am also getting same slowdown on GeForce RTX 2080 Ti</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">Posted here first.<br>
<a href="https://discuss.pytorch.org/t/jited-gru-too-slow/68873" rel="nofollow">https://discuss.pytorch.org/t/jited-gru-too-slow/68873</a></p>
<p dir="auto">Warm starting does not change much at all.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/suo/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/suo">@suo</a></p>