<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">There is a weird behaviour of a backward function when performing a reduction operation (sum) on a dense tensor generated from the sparse one. For example, just multiplying the dense tensor by one causes the generation of the RuntimeError during the backward pass: <code class="notranslate">view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.</code>. Weirdly enough if there are several sums for resulting dense tensor the problem disappears.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch


idxs = torch.tensor([[0, 2, 3], [1, 1, 2], [2, 1, 4], [3, 5, 1]], device=0)
values = torch.randn((4, 6, 5), device=0, requires_grad=True)

sparse_tensor = torch.sparse_coo_tensor(indices=idxs.t(),
                                        values=values[idxs.split(split_size=1, dim=1)].squeeze(),
                                        size=values.shape)
dense_tensor = torch.sparse.sum(sparse_tensor, dim=2).to_dense()
dense_tensor = dense_tensor.sum(dim=1) # + dense_tensor.sum(dim=1)

(dense_tensor * 1).sum().backward()  #  `view size is not compatible with input tensor's size and stride`
# dense_tensor.sum().backward()  #  no exceptions observed"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>


<span class="pl-s1">idxs</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">tensor</span>([[<span class="pl-c1">0</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>], [<span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>], [<span class="pl-c1">2</span>, <span class="pl-c1">1</span>, <span class="pl-c1">4</span>], [<span class="pl-c1">3</span>, <span class="pl-c1">5</span>, <span class="pl-c1">1</span>]], <span class="pl-s1">device</span><span class="pl-c1">=</span><span class="pl-c1">0</span>)
<span class="pl-s1">values</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">randn</span>((<span class="pl-c1">4</span>, <span class="pl-c1">6</span>, <span class="pl-c1">5</span>), <span class="pl-s1">device</span><span class="pl-c1">=</span><span class="pl-c1">0</span>, <span class="pl-s1">requires_grad</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)

<span class="pl-s1">sparse_tensor</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">sparse_coo_tensor</span>(<span class="pl-s1">indices</span><span class="pl-c1">=</span><span class="pl-s1">idxs</span>.<span class="pl-en">t</span>(),
                                        <span class="pl-s1">values</span><span class="pl-c1">=</span><span class="pl-s1">values</span>[<span class="pl-s1">idxs</span>.<span class="pl-en">split</span>(<span class="pl-s1">split_size</span><span class="pl-c1">=</span><span class="pl-c1">1</span>, <span class="pl-s1">dim</span><span class="pl-c1">=</span><span class="pl-c1">1</span>)].<span class="pl-en">squeeze</span>(),
                                        <span class="pl-s1">size</span><span class="pl-c1">=</span><span class="pl-s1">values</span>.<span class="pl-s1">shape</span>)
<span class="pl-s1">dense_tensor</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-s1">sparse</span>.<span class="pl-en">sum</span>(<span class="pl-s1">sparse_tensor</span>, <span class="pl-s1">dim</span><span class="pl-c1">=</span><span class="pl-c1">2</span>).<span class="pl-en">to_dense</span>()
<span class="pl-s1">dense_tensor</span> <span class="pl-c1">=</span> <span class="pl-s1">dense_tensor</span>.<span class="pl-en">sum</span>(<span class="pl-s1">dim</span><span class="pl-c1">=</span><span class="pl-c1">1</span>) <span class="pl-c"># + dense_tensor.sum(dim=1)</span>

(<span class="pl-s1">dense_tensor</span> <span class="pl-c1">*</span> <span class="pl-c1">1</span>).<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()  <span class="pl-c">#  `view size is not compatible with input tensor's size and stride`</span>
<span class="pl-c"># dense_tensor.sum().backward()  #  no exceptions observed</span></pre></div>
<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch Version is 1.2.0<br>
OS: Ubuntu<br>
Cuda: 10.1</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/vincentqb/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/vincentqb">@vincentqb</a></p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">Some PyTorch primitives expect the gradient passed in during the backward pass to be contiguous, but not all functions produce a contiguous gradient in their backward pass. When two incompatible functions -- one which returns a non-contiguous gradient in the backward pass, and another which expects a contiguous gradient as input to its backward pass -- are strung together, the autodifferentiation fails.<br>
In particular, <code class="notranslate">permute</code> and <code class="notranslate">pad</code> don’t play well together, as the following example shows:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="# short example to reproduce the error:

import torch
import torch.nn.functional as F

inv = torch.zeros((3, 8), dtype=torch.float).requires_grad_()
indices = torch.zeros((2, 3), dtype=torch.long)

comb = torch.sparse.FloatTensor(indices, inv, (4, 4, 8)).to_dense()
big = F.pad(comb, (0, 0, 1, 1, 1, 1))
shaped = big.view(-1, 8).permute(1, 0).unsqueeze(0)
res = F.fold(shaped, output_size=(5, 5),
             kernel_size=(2, 2), padding=(1, 1))
res.sum().backward()"><pre class="notranslate"><span class="pl-c"># short example to reproduce the error:</span>

<span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-k">import</span> <span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">functional</span> <span class="pl-k">as</span> <span class="pl-v">F</span>

<span class="pl-s1">inv</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">zeros</span>((<span class="pl-c1">3</span>, <span class="pl-c1">8</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">float</span>).<span class="pl-en">requires_grad_</span>()
<span class="pl-s1">indices</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">zeros</span>((<span class="pl-c1">2</span>, <span class="pl-c1">3</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">long</span>)

<span class="pl-s1">comb</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-s1">sparse</span>.<span class="pl-v">FloatTensor</span>(<span class="pl-s1">indices</span>, <span class="pl-s1">inv</span>, (<span class="pl-c1">4</span>, <span class="pl-c1">4</span>, <span class="pl-c1">8</span>)).<span class="pl-en">to_dense</span>()
<span class="pl-s1">big</span> <span class="pl-c1">=</span> <span class="pl-v">F</span>.<span class="pl-en">pad</span>(<span class="pl-s1">comb</span>, (<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>))
<span class="pl-s1">shaped</span> <span class="pl-c1">=</span> <span class="pl-s1">big</span>.<span class="pl-en">view</span>(<span class="pl-c1">-</span><span class="pl-c1">1</span>, <span class="pl-c1">8</span>).<span class="pl-en">permute</span>(<span class="pl-c1">1</span>, <span class="pl-c1">0</span>).<span class="pl-en">unsqueeze</span>(<span class="pl-c1">0</span>)
<span class="pl-s1">res</span> <span class="pl-c1">=</span> <span class="pl-v">F</span>.<span class="pl-en">fold</span>(<span class="pl-s1">shaped</span>, <span class="pl-s1">output_size</span><span class="pl-c1">=</span>(<span class="pl-c1">5</span>, <span class="pl-c1">5</span>),
             <span class="pl-s1">kernel_size</span><span class="pl-c1">=</span>(<span class="pl-c1">2</span>, <span class="pl-c1">2</span>), <span class="pl-s1">padding</span><span class="pl-c1">=</span>(<span class="pl-c1">1</span>, <span class="pl-c1">1</span>))
<span class="pl-s1">res</span>.<span class="pl-en">sum</span>().<span class="pl-en">backward</span>()</pre></div>
<p dir="auto">which results in</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead. (view at ../aten/src/ATen/native/TensorShape.cpp:1185)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) + 0x65 (0x7f0ab4b2eb15 in /usr/people/bnehoran/pytorch/torch/lib/libc10.so)
frame #1: at::native::view(at::Tensor const&amp;, c10::ArrayRef&lt;long&gt;) + 0x2fb (0x7f0ac8ce33cb in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #2: &lt;unknown function&gt; + 0x118f433 (0x7f0ac8edf433 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #3: &lt;unknown function&gt; + 0x11dfb17 (0x7f0ac8f2fb17 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #4: &lt;unknown function&gt; + 0x1054db3 (0x7f0ac8da4db3 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #5: at::native::sparse_mask_cpu(at::Tensor const&amp;, at::Tensor const&amp;) + 0x84 (0x7f0ac8da6614 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #6: &lt;unknown function&gt; + 0x126c8e3 (0x7f0ac8fbc8e3 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #7: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #8: at::Tensor c10::Dispatcher::callUnboxed&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::OperatorHandle const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xb2 (0x7f0acdfcac12 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #9: &lt;unknown function&gt; + 0x2d94771 (0x7f0acaae4771 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #10: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #11: at::native::to_dense_backward(at::Tensor const&amp;, at::Tensor const&amp;) + 0x21a (0x7f0ac8ca7b8a in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #12: &lt;unknown function&gt; + 0x12a4f53 (0x7f0ac8ff4f53 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #13: &lt;unknown function&gt; + 0x2b1a4be (0x7f0aca86a4be in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #14: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #15: at::Tensor c10::Dispatcher::callUnboxed&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::OperatorHandle const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xb2 (0x7f0acdfcac12 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #16: torch::autograd::generated::ToDenseBackward::apply(std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt;&amp;&amp;) + 0xf9 (0x7f0aca6a06f9 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #17: &lt;unknown function&gt; + 0x2feebdb (0x7f0acad3ebdb in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #18: torch::autograd::Engine::evaluate_function(std::shared_ptr&lt;torch::autograd::GraphTask&gt;&amp;, torch::autograd::Node*, torch::autograd::InputBuffer&amp;) + 0x16ac (0x7f0acad3a34c in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #19: torch::autograd::Engine::thread_main(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;, bool) + 0x591 (0x7f0acad3b511 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #20: torch::autograd::Engine::thread_init(int) + 0x49 (0x7f0acad32b89 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #21: torch::autograd::python::PythonEngine::thread_init(int) + 0x48 (0x7f0ace244cc8 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #22: &lt;unknown function&gt; + 0xc819d (0x7f0af8e5819d in /usr/people/bnehoran/anaconda3/envs/pytorch/lib/libstdc++.so.6)
frame #23: &lt;unknown function&gt; + 0x76ba (0x7f0b15f766ba in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #24: clone + 0x6d (0x7f0b15cac41d in /lib/x86_64-linux-gnu/libc.so.6)"><pre class="notranslate"><code class="notranslate">RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead. (view at ../aten/src/ATen/native/TensorShape.cpp:1185)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) + 0x65 (0x7f0ab4b2eb15 in /usr/people/bnehoran/pytorch/torch/lib/libc10.so)
frame #1: at::native::view(at::Tensor const&amp;, c10::ArrayRef&lt;long&gt;) + 0x2fb (0x7f0ac8ce33cb in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #2: &lt;unknown function&gt; + 0x118f433 (0x7f0ac8edf433 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #3: &lt;unknown function&gt; + 0x11dfb17 (0x7f0ac8f2fb17 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #4: &lt;unknown function&gt; + 0x1054db3 (0x7f0ac8da4db3 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #5: at::native::sparse_mask_cpu(at::Tensor const&amp;, at::Tensor const&amp;) + 0x84 (0x7f0ac8da6614 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #6: &lt;unknown function&gt; + 0x126c8e3 (0x7f0ac8fbc8e3 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #7: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #8: at::Tensor c10::Dispatcher::callUnboxed&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::OperatorHandle const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xb2 (0x7f0acdfcac12 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #9: &lt;unknown function&gt; + 0x2d94771 (0x7f0acaae4771 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #10: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #11: at::native::to_dense_backward(at::Tensor const&amp;, at::Tensor const&amp;) + 0x21a (0x7f0ac8ca7b8a in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #12: &lt;unknown function&gt; + 0x12a4f53 (0x7f0ac8ff4f53 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #13: &lt;unknown function&gt; + 0x2b1a4be (0x7f0aca86a4be in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #14: &lt;unknown function&gt; + 0x11decb4 (0x7f0ac8f2ecb4 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #15: at::Tensor c10::Dispatcher::callUnboxed&lt;at::Tensor, at::Tensor const&amp;, at::Tensor const&amp;&gt;(c10::OperatorHandle const&amp;, at::Tensor const&amp;, at::Tensor const&amp;) const + 0xb2 (0x7f0acdfcac12 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #16: torch::autograd::generated::ToDenseBackward::apply(std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt;&amp;&amp;) + 0xf9 (0x7f0aca6a06f9 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #17: &lt;unknown function&gt; + 0x2feebdb (0x7f0acad3ebdb in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #18: torch::autograd::Engine::evaluate_function(std::shared_ptr&lt;torch::autograd::GraphTask&gt;&amp;, torch::autograd::Node*, torch::autograd::InputBuffer&amp;) + 0x16ac (0x7f0acad3a34c in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #19: torch::autograd::Engine::thread_main(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;, bool) + 0x591 (0x7f0acad3b511 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #20: torch::autograd::Engine::thread_init(int) + 0x49 (0x7f0acad32b89 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_cpu.so)
frame #21: torch::autograd::python::PythonEngine::thread_init(int) + 0x48 (0x7f0ace244cc8 in /usr/people/bnehoran/pytorch/torch/lib/libtorch_python.so)
frame #22: &lt;unknown function&gt; + 0xc819d (0x7f0af8e5819d in /usr/people/bnehoran/anaconda3/envs/pytorch/lib/libstdc++.so.6)
frame #23: &lt;unknown function&gt; + 0x76ba (0x7f0b15f766ba in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #24: clone + 0x6d (0x7f0b15cac41d in /lib/x86_64-linux-gnu/libc.so.6)
</code></pre></div>
<p dir="auto">Tested on the master branch (<code class="notranslate">1.5.0a0+7fdc6cb</code>) but was already present in versions as early as 1.2.</p>
<p dir="auto"><em>Edit: The error in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="512494506" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/28650" data-hovercard-type="issue" data-hovercard-url="/pytorch/pytorch/issues/28650/hovercard" href="https://github.com/pytorch/pytorch/issues/28650">#28650</a> might be related to this issue.</em></p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a></p>