<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>

<p dir="auto">I divide an element by zero and get an inf. But this inf is not used in the following calculation. However, after backward, I still get a nan gradient for this element.</p>
<h2 dir="auto">To Reproduce</h2>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch
divide_tensor = torch.Tensor([1.0, 0.0, 1.0])
weight = torch.ones(3)
weight.requires_grad = True
tmp = weight / divide_tensor
loss = tmp[0]
loss.backward()
print(weight.grad)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-s1">divide_tensor</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-v">Tensor</span>([<span class="pl-c1">1.0</span>, <span class="pl-c1">0.0</span>, <span class="pl-c1">1.0</span>])
<span class="pl-s1">weight</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>(<span class="pl-c1">3</span>)
<span class="pl-s1">weight</span>.<span class="pl-s1">requires_grad</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span>
<span class="pl-s1">tmp</span> <span class="pl-c1">=</span> <span class="pl-s1">weight</span> <span class="pl-c1">/</span> <span class="pl-s1">divide_tensor</span>
<span class="pl-s1">loss</span> <span class="pl-c1">=</span> <span class="pl-s1">tmp</span>[<span class="pl-c1">0</span>]
<span class="pl-s1">loss</span>.<span class="pl-en">backward</span>()
<span class="pl-en">print</span>(<span class="pl-s1">weight</span>.<span class="pl-s1">grad</span>)</pre></div>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">I expect gradient [1., 0., 0.] because second and third elements of weight are not related to loss.</p>
<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch version: 1.0.1<br>
Is debug build: No<br>
CUDA used to build PyTorch: 9.0.176</p>
<p dir="auto">OS: Ubuntu 16.04.5 LTS<br>
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609<br>
CMake version: version 3.5.1</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: Yes<br>
CUDA runtime version: 9.0.176<br>
GPU models and configuration:<br>
GPU 0: TITAN Xp<br>
GPU 1: TITAN Xp<br>
GPU 2: TITAN Xp<br>
GPU 3: TITAN Xp</p>
<p dir="auto">Nvidia driver version: 410.78<br>
cuDNN version: Could not collect</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip] numpy==1.16.4<br>
[pip] torch==1.0.1<br>
[pip] torchvision==0.2.1<br>
[conda] _tflow_select             2.3.0                       mkl<br>
[conda] blas                      1.0                         mkl<br>
[conda] mkl                       2019.4                      243<br>
[conda] mkl-service               2.0.2            py36h7b6447c_0<br>
[conda] mkl_fft                   1.0.12           py36ha843d7b_0<br>
[conda] mkl_random                1.0.2            py36hd81dba3_0<br>
[conda] pytorch                   1.0.1           cuda90py36h8b0c50b_0<br>
[conda] tensorflow                1.13.1          mkl_py36h27d456a_0<br>
[conda] tensorflow-base           1.13.1          mkl_py36h7ce6ba3_0<br>
[conda] torchvision               0.2.1                    py36_0</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a></p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto"><code class="notranslate">nan</code> propagates through computations during a <code class="notranslate">backward()</code> pass even if it wasn't accessed at all.</p>
<h2 dir="auto">To reproduce</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="a = torch.arange(4, dtype=torch.float, requires_grad=True)
a.log()[-1].backward()
print(a.grad)"><pre class="notranslate"><code class="notranslate">a = torch.arange(4, dtype=torch.float, requires_grad=True)
a.log()[-1].backward()
print(a.grad)
</code></pre></div>
<p dir="auto">prints <code class="notranslate">tensor([   nan, 0.0000, 0.0000, 0.3333])</code></p>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">It should print <code class="notranslate">tensor([0.0000, 0.0000, 0.0000, 0.3333])</code>, like the following code does:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="a = torch.arange(4, dtype=torch.float, requires_grad=True)
a[-1].log().backward()
print(a.grad)"><pre class="notranslate"><code class="notranslate">a = torch.arange(4, dtype=torch.float, requires_grad=True)
a[-1].log().backward()
print(a.grad)
</code></pre></div>
<p dir="auto">The same should be true if <code class="notranslate">[-1]</code> is replaced with <code class="notranslate">.max()</code> (which causes the same entry to be accessed, in this particular example), which is how I originally encountered the issue.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Collecting environment information...
PyTorch version: 1.0.0a0+4c11dee
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Mac OSX 10.14.2
GCC version: Could not collect
CMake version: version 3.13.0

Python version: 3.6
Is CUDA available: No
CUDA runtime version: 10.0.130
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/local/cuda/lib/libcudnn.7.dylib
/usr/local/cuda/lib/libcudnn.dylib
/usr/local/cuda/lib/libcudnn_static.a

Versions of relevant libraries:
[pip] Could not collect
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.0                    &lt;pip&gt;
[conda] mkl                       2018.0.3                      1  
[conda] mkl-include               2019.0                    &lt;pip&gt;
[conda] mkl-service               1.1.2            py36h6b9c3cc_4  
[conda] mkl_fft                   1.0.2            py36h6b9c3cc_0  
[conda] mkl_random                1.0.1            py36h5d10147_1  
[conda] torch                     1.0.0a0+4c11dee           &lt;pip&gt;
[conda] torchfile                 0.1.0                     &lt;pip&gt;
[conda] torchvision               0.2.0                     &lt;pip&gt;"><pre class="notranslate"><code class="notranslate">Collecting environment information...
PyTorch version: 1.0.0a0+4c11dee
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Mac OSX 10.14.2
GCC version: Could not collect
CMake version: version 3.13.0

Python version: 3.6
Is CUDA available: No
CUDA runtime version: 10.0.130
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/local/cuda/lib/libcudnn.7.dylib
/usr/local/cuda/lib/libcudnn.dylib
/usr/local/cuda/lib/libcudnn_static.a

Versions of relevant libraries:
[pip] Could not collect
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.0                    &lt;pip&gt;
[conda] mkl                       2018.0.3                      1  
[conda] mkl-include               2019.0                    &lt;pip&gt;
[conda] mkl-service               1.1.2            py36h6b9c3cc_4  
[conda] mkl_fft                   1.0.2            py36h6b9c3cc_0  
[conda] mkl_random                1.0.1            py36h5d10147_1  
[conda] torch                     1.0.0a0+4c11dee           &lt;pip&gt;
[conda] torchfile                 0.1.0                     &lt;pip&gt;
[conda] torchvision               0.2.0                     &lt;pip&gt;
</code></pre></div>