<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">Per <a href="https://pytorch.org/docs/stable/nn.html#maxpool2d" rel="nofollow">documentation of MaxPool</a>, input is "implicitly zero-padded on both sides for padding number of points." The implementation differs by implicitly padding with identity elements, rather than zero.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Create a tensor of signed type, filled with negative values.</li>
<li>Invoke a MaxPool with positive padding.</li>
<li>Compare against ground truth, either computed by hand or created by explicit zero-padding.</li>
</ol>
<p dir="auto">Minimal code sample / unit test:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# Maxpool of non-positive values, such that padding is included in the computation of each result element.
import torch
r = torch.nn.functional.max_pool2d(-torch.rand(2,2,64), kernel_size=[3,3], padding=1)
assert torch.equal(r, torch.zeros_like(r))"><pre class="notranslate"><code class="notranslate"># Maxpool of non-positive values, such that padding is included in the computation of each result element.
import torch
r = torch.nn.functional.max_pool2d(-torch.rand(2,2,64), kernel_size=[3,3], padding=1)
assert torch.equal(r, torch.zeros_like(r))
</code></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">According to documentation, the values near the edge will be computed using the implicit zero-padding. The maximum of a negative number and zero is zero, so such values <em>ought</em> to be zero. In the provided example, the assert should not fire.</p>
<p dir="auto">When the code sample above is run, however, the assert fires.</p>
<h2 dir="auto">Environment</h2>
<p dir="auto">Information from collect_env.py:</p>
<ul dir="auto">
<li>
<p dir="auto">PyTorch version: 1.5.1</p>
</li>
<li>
<p dir="auto">Is debug build: No</p>
</li>
<li>
<p dir="auto">CUDA used to build PyTorch: 10.2</p>
</li>
<li>
<p dir="auto">OS: Microsoft Windows 10 Home</p>
</li>
<li>
<p dir="auto">GCC version: (x86_64-win32-seh-rev0, Built by MinGW-W64 project) 8.1.0</p>
</li>
<li>
<p dir="auto">CMake version: version 3.14.1</p>
</li>
<li>
<p dir="auto">Python version: 3.8</p>
</li>
<li>
<p dir="auto">Is CUDA available: Yes</p>
</li>
<li>
<p dir="auto">CUDA runtime version: Could not collect</p>
</li>
<li>
<p dir="auto">GPU models and configuration: GPU 0: GeForce GTX 960M</p>
</li>
<li>
<p dir="auto">Nvidia driver version: 442.50</p>
</li>
<li>
<p dir="auto">cuDNN version: Could not collect</p>
</li>
<li>
<p dir="auto">Versions of relevant libraries:</p>
</li>
<li>
<p dir="auto">[pip] Could not collect</p>
</li>
<li>
<p dir="auto">[conda] Could not collect</p>
</li>
</ul>
<h2 dir="auto">Additional context</h2>
<p dir="auto">N/A</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jlin27/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jlin27">@jlin27</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mruberry/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mruberry">@mruberry</a></p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">According to the documentation on <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.MaxPool1d" rel="nofollow"><code class="notranslate">nn.MaxPool1d</code></a>:</p>
<blockquote>
<p dir="auto">If <code class="notranslate">padding</code> is non-zero, then <strong>the input is implicitly zero-padded on both sides</strong> for <code class="notranslate">padding</code> number of points. <code class="notranslate">dilation</code> controls the spacing between the kernel points. It is harder to describe, but this link has a nice visualization of what <code class="notranslate">dilation</code> does.</p>
</blockquote>
<p dir="auto">However, the input is never padded with zeros, either explicitly or implicitly. Doing a bit of source diving I found that the maximum operation <a href="https://github.com/pytorch/pytorch/blob/d35a4c202e917b5d5cb67ac749fcf7931212c25b/aten/src/ATen/native/DilatedMaxPool2d.cpp#L59">initializes with negative infinity</a>, not zero, and only considers entries from the original, unpadded input tensor. Therefore it would be correct to say that the max-pooling operation uses implicit negative infinity padding but not zero-padding.</p>
<p dir="auto">This appears to be either a bug in the API or documentation (of course PEBCAK is always a possibility).</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Install PyTorch</li>
<li>Run the following code</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import torch
&gt;&gt;&gt; from torch.nn.functional import max_pool1d
&gt;&gt;&gt; x = torch.tensor([[[-1., -2.]]])
&gt;&gt;&gt; max_pool1d(x, kernel_size=2, padding=1)
tensor([[[-1., -2.]]])
&gt;&gt;&gt; max_pool1d(x.cuda(), kernel_size=2, padding=1)
tensor([[[-1., -2.]]], device='cuda:0')"><pre class="notranslate"><code class="notranslate">&gt;&gt;&gt; import torch
&gt;&gt;&gt; from torch.nn.functional import max_pool1d
&gt;&gt;&gt; x = torch.tensor([[[-1., -2.]]])
&gt;&gt;&gt; max_pool1d(x, kernel_size=2, padding=1)
tensor([[[-1., -2.]]])
&gt;&gt;&gt; max_pool1d(x.cuda(), kernel_size=2, padding=1)
tensor([[[-1., -2.]]], device='cuda:0')
</code></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">If zero-padding was being used we would expect the output to be <code class="notranslate">tensor([[[0., 0.]]])</code> since zero is larger than all the input tensor elements.</p>
<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch version: 1.4.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.1</p>
<p dir="auto">OS: Red Hat Enterprise Linux Workstation release 7.7 (Maipo)<br>
GCC version: (GCC) 6.3.0<br>
CMake version: version 3.11.4</p>
<p dir="auto">Python version: 3.7<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.0.130<br>
GPU models and configuration:<br>
GPU 0: GeForce RTX 2080 Ti<br>
GPU 1: GeForce RTX 2080 Ti</p>
<p dir="auto">Nvidia driver version: 430.40<br>
cuDNN version: 7.6.3</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] numpy==1.18.1<br>
[pip3] torch==1.4.0<br>
[pip3] torchvision==0.5.0<br>
[conda] Could not collect</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">This issue is based on a <a href="https://stackoverflow.com/questions/60240434" rel="nofollow">question originally asked on StackOverflow</a> by user trsvchn.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jlin27/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jlin27">@jlin27</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mruberry/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mruberry">@mruberry</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/heitorschueroff/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/heitorschueroff">@heitorschueroff</a></p>