<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">I think the vector strides are passed incorrectly to <code class="notranslate">gemv</code> on GPU. It works well on CPU.</p>
<h2 dir="auto">To Reproduce</h2>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch

mat = torch.randn(2, 2).cuda()
vec = torch.randn(2).cuda().requires_grad_(True)
(mat @ vec).sum().backward()"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>

<span class="pl-s1">mat</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">randn</span>(<span class="pl-c1">2</span>, <span class="pl-c1">2</span>).<span class="pl-en">cuda</span>()
<span class="pl-s1">vec</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">randn</span>(<span class="pl-c1">2</span>).<span class="pl-en">cuda</span>().<span class="pl-en">requires_grad_</span>(<span class="pl-c1">True</span>)
(<span class="pl-s1">mat</span> @ <span class="pl-s1">vec</span>).<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()</pre></div>
<p dir="auto">Here I get:</p>
<blockquote>
<p dir="auto">RuntimeError: at::cuda::blas::gemv argument incx must be positive and less than 2147483647 but got 0</p>
</blockquote>
<p dir="auto">Originally I found the bug as an issue during DGER in the gradient of <code class="notranslate">mat @ vec</code>, in the generated function <code class="notranslate">MvBackward::apply</code> in <code class="notranslate">torch/csrc/autograd/generated/Functions.cpp</code>. The error message is the same as <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="477959921" data-permission-text="Title is private" data-url="https://github.com/cornellius-gp/gpytorch/issues/834" data-hovercard-type="issue" data-hovercard-url="/cornellius-gp/gpytorch/issues/834/hovercard" href="https://github.com/cornellius-gp/gpytorch/issues/834">cornellius-gp/gpytorch#834</a> .  But, attempting to write minimal code to reproduce that, I hit this other bug instead.</p>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">The gradient for <code class="notranslate">vec</code> should be calculated, and the program should not crash.</p>
<h2 dir="auto">Environment</h2>
<p dir="auto">I have reproduced this bug in <code class="notranslate">master</code>, and Pytorch 1.4.</p>
<details>
<summary> Details of master version</summary>
<blockquote>
<p dir="auto">PyTorch version: 1.6.0a0+cf82011<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.0</p>
<p dir="auto">OS: Ubuntu 16.04.6 LTS<br>
GCC version: (Ubuntu 7.1.0-10ubuntu1~16.04.york0) 7.1.0<br>
CMake version: version 3.14.0</p>
<p dir="auto">Python version: 3.7<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.0.130<br>
GPU models and configuration:<br>
GPU 0: GeForce GTX 1080<br>
GPU 1: GeForce GT 710</p>
<p dir="auto">Nvidia driver version: 418.87.01<br>
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] numpy==1.11.0<br>
[conda] blas                      1.0                         mkl<br>
[conda] gpytorch                  1.0.1                     dev_0    <br>
[conda] magma-cuda100             2.5.2                         1    pytorch<br>
[conda] mkl                       2020.0                      166<br>
[conda] mkl-include               2020.1                      217<br>
[conda] mkl-service               2.3.0            py37he904b0f_0<br>
[conda] mkl_fft                   1.0.15           py37ha843d7b_0<br>
[conda] mkl_random                1.1.0            py37hd6b4f25_0<br>
[conda] numpy                     1.18.1           py37h4f9e942_0<br>
[conda] numpy-base                1.18.1           py37hde5b4d6_1<br>
[conda] numpydoc                  0.9.2                      py_0<br>
[conda] torch                     1.6.0a0+cf82011          pypi_0    pypi<br>
[conda] torchvision               0.6.0                    pypi_0    pypi</p>
</blockquote>
</details>
<details>
<summary>Details of 1.4.0 version</summary>
<blockquote>
<p dir="auto">PyTorch version: 1.4.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.1</p>
<p dir="auto">OS: Ubuntu 16.04.6 LTS<br>
GCC version: (Ubuntu 7.1.0-10ubuntu1~16.04.york0) 7.1.0<br>
CMake version: version 3.16.0-rc3</p>
<p dir="auto">Python version: 3.7<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.1.243<br>
GPU models and configuration:<br>
GPU 0: GeForce GTX 1080<br>
GPU 1: GeForce GT 710</p>
<p dir="auto">Nvidia driver version: 418.87.01<br>
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] gpytorch==1.0.1<br>
[pip3] numpy==1.18.2<br>
[pip3] torch==1.4.0<br>
[pip3] torchvision==0.5.0<br>
[conda] Could not collect</p>
</blockquote>
</details>
<h2 dir="auto">Additional context</h2>
<p dir="auto">I would fix this myself but the C++ call stack is too deep and I get lost. I also don't know where the <code class="notranslate">generated/</code> functions come from. Thus I pass the baton to you, and will use a workaround.</p>
<h2 dir="auto">Workaround</h2>
<p dir="auto">Use <code class="notranslate">vec.unsqueeze(-1)</code> to turn it into a <code class="notranslate">gemm</code>. Squeeze the result.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a></p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>

<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch
import torch.nn as nn

torch.set_default_tensor_type('torch.cuda.FloatTensor')
x = nn.Parameter(torch.ones(2, 2))
(x @ torch.ones(2)).sum().backward()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-11-c3b66f275e9a&gt; in &lt;module&gt;()
      1 x = nn.Parameter(torch.ones(2, 2))
----&gt; 2 (x @ torch.ones(2)).sum().backward()

1 frames
/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     97     Variable._execution_engine.run_backward(
     98         tensors, grad_tensors, retain_graph, create_graph,
---&gt; 99         allow_unreachable=True)  # allow_unreachable flag
    100 
    101 

RuntimeError: cublas runtime error : an invalid numeric value was used as an argument at /pytorch/aten/src/THC/THCBlas.cu:120"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-k">import</span> <span class="pl-s1">torch</span>.<span class="pl-s1">nn</span> <span class="pl-k">as</span> <span class="pl-s1">nn</span>

<span class="pl-s1">torch</span>.<span class="pl-en">set_default_tensor_type</span>(<span class="pl-s">'torch.cuda.FloatTensor'</span>)
<span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">nn</span>.<span class="pl-v">Parameter</span>(<span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>, <span class="pl-c1">2</span>))
(<span class="pl-s1">x</span> @ <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>)).<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span>
<span class="pl-v">RuntimeError</span>                              <span class="pl-v">Traceback</span> (<span class="pl-s1">most</span> <span class="pl-s1">recent</span> <span class="pl-s1">call</span> <span class="pl-s1">last</span>)
<span class="pl-c1">&lt;</span><span class="pl-s1">ipython</span><span class="pl-c1">-</span><span class="pl-s1">input</span><span class="pl-c1">-</span><span class="pl-c1">11</span><span class="pl-c1">-</span><span class="pl-s1">c3b66f275e9a</span><span class="pl-c1">&gt;</span> <span class="pl-s1">in</span> <span class="pl-c1">&lt;</span><span class="pl-s1">module</span><span class="pl-c1">&gt;</span>()
      <span class="pl-c1">1</span> <span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">nn</span>.<span class="pl-v">Parameter</span>(<span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>, <span class="pl-c1">2</span>))
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">2</span> (<span class="pl-s1">x</span> @ <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>)).<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()

<span class="pl-c1">1</span> <span class="pl-s1">frames</span>
<span class="pl-c1">/</span><span class="pl-s1">usr</span><span class="pl-c1">/</span><span class="pl-s1">local</span><span class="pl-c1">/</span><span class="pl-s1">lib</span><span class="pl-c1">/</span><span class="pl-s1">python3</span>.<span class="pl-c1">6</span><span class="pl-c1">/</span><span class="pl-s1">dist</span><span class="pl-c1">-</span><span class="pl-s1">packages</span><span class="pl-c1">/</span><span class="pl-s1">torch</span><span class="pl-c1">/</span><span class="pl-s1">autograd</span><span class="pl-c1">/</span><span class="pl-s1">__init__</span>.<span class="pl-s1">py</span> <span class="pl-c1">in</span> <span class="pl-en">backward</span>(<span class="pl-s1">tensors</span>, <span class="pl-s1">grad_tensors</span>, <span class="pl-s1">retain_graph</span>, <span class="pl-s1">create_graph</span>, <span class="pl-s1">grad_variables</span>)
     <span class="pl-c1">97</span>     <span class="pl-v">Variable</span>.<span class="pl-s1">_execution_engine</span>.<span class="pl-en">run_backward</span>(
     <span class="pl-c1">98</span>         <span class="pl-s1">tensors</span>, <span class="pl-s1">grad_tensors</span>, <span class="pl-s1">retain_graph</span>, <span class="pl-s1">create_graph</span>,
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">99</span>         <span class="pl-s1">allow_unreachable</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)  <span class="pl-c"># allow_unreachable flag</span>
    <span class="pl-c1">100</span> 
    <span class="pl-c1">101</span> 

<span class="pl-v">RuntimeError</span>: <span class="pl-s1">cublas</span> <span class="pl-s1">runtime</span> <span class="pl-s1">error</span> : <span class="pl-s1">an</span> <span class="pl-s1">invalid</span> <span class="pl-s1">numeric</span> <span class="pl-s1">value</span> <span class="pl-s1">was</span> <span class="pl-s1">used</span> <span class="pl-k">as</span> <span class="pl-s1">an</span> <span class="pl-s1">argument</span> <span class="pl-s1">at</span> <span class="pl-c1">/</span><span class="pl-s1">pytorch</span><span class="pl-c1">/</span><span class="pl-s1">aten</span><span class="pl-c1">/</span><span class="pl-s1">src</span><span class="pl-c1">/</span><span class="pl-v">THC</span><span class="pl-c1">/</span><span class="pl-v">THCBlas</span>.<span class="pl-s1">cu</span>:<span class="pl-c1">120</span></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">No exception is raised.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Collecting environment information...
PyTorch version: 1.3.1+cu100
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.12.0

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.3

Versions of relevant libraries:
[pip3] numpy==1.17.4
[pip3] torch==1.3.1+cu100
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.4.2+cu100
[conda] Could not collect"><pre class="notranslate"><code class="notranslate">Collecting environment information...
PyTorch version: 1.3.1+cu100
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.12.0

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.3

Versions of relevant libraries:
[pip3] numpy==1.17.4
[pip3] torch==1.3.1+cu100
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.4.2+cu100
[conda] Could not collect
</code></pre></div>
<p dir="auto">This code was run on Google Colab. The bug also reproduces on my server (Ubuntu 16.04LTS, 4x GTX1080Ti GPUs).</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">The bug doesn't occur if:</p>
<ol dir="auto">
<li>If I replace <code class="notranslate">sum()</code> with <code class="notranslate">mean()</code></li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="(x @ torch.ones(2)).mean().backward()"><pre class="notranslate">(<span class="pl-s1">x</span> @ <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>)).<span class="pl-en">mean</span>().<span class="pl-en">backward</span>()</pre></div>
<ol start="2" dir="auto">
<li>If I use a different shape of the tensor</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="(x @ torch.ones(2, 1)).sum().backward()"><pre class="notranslate">(<span class="pl-s1">x</span> @ <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">2</span>, <span class="pl-c1">1</span>)).<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()</pre></div>
<ol start="3" dir="auto">
<li>If I run the code on CPU</li>
</ol>
<p dir="auto">Therefore, I believe that this is not intended behavior.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jerryzh168/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jerryzh168">@jerryzh168</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a></p>