<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>:yes</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: Linus centos 7</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: pip</li>
<li><strong>TensorFlow version (use command below)</strong>: 1.4</li>
<li><strong>Python version</strong>: 2.7.5</li>
<li><strong>Bazel version (if compiling from source)</strong>: 0</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>:0</li>
<li><strong>CUDA/cuDNN version</strong>:0</li>
<li><strong>GPU model and memory</strong>:0</li>
<li><strong>Exact command to reproduce</strong>:</li>
</ul>
<p dir="auto">In the following situation, TF <a href="https://www.tensorflow.org/api_guides/python/nn#Convolution" rel="nofollow">doc</a> is not correct.</p>
<ul dir="auto">
<li>Input tensor shape : [1, 5, 2, 1]</li>
<li>Kernel shape:           [1, 3, 1, 1]</li>
<li>Stride :                      [1, 5, 5, 1]</li>
<li>Padding =                  "SAME"</li>
</ul>
<p dir="auto">According to the formula we can compute :<br>
out_h = 1<br>
out_w = 1</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="if (in_height % strides[1] == 0):
  pad_along_height = max(filter_height - strides[1], 0)
else:
  pad_along_height = max(filter_height - (in_height % strides[1]), 0)
if (in_width % strides[2] == 0):
  pad_along_width = max(filter_width - strides[2], 0)
else:
  pad_along_width = max(filter_width - (in_width % strides[2]), 0)"><pre class="notranslate"><code class="notranslate">if (in_height % strides[1] == 0):
  pad_along_height = max(filter_height - strides[1], 0)
else:
  pad_along_height = max(filter_height - (in_height % strides[1]), 0)
if (in_width % strides[2] == 0):
  pad_along_width = max(filter_width - strides[2], 0)
else:
  pad_along_width = max(filter_width - (in_width % strides[2]), 0)
</code></pre></div>
<p dir="auto">gives :<br>
pad_along_height = 0<br>
pad_along_width = 1</p>
<p dir="auto">then</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="pad_top = pad_along_height // 2
pad_bottom = pad_along_height - pad_top
pad_left = pad_along_width // 2
pad_right = pad_along_width - pad_left"><pre class="notranslate"><code class="notranslate">pad_top = pad_along_height // 2
pad_bottom = pad_along_height - pad_top
pad_left = pad_along_width // 2
pad_right = pad_along_width - pad_left
</code></pre></div>
<p dir="auto">gives:</p>
<p dir="auto">pad_top = 0<br>
pad_bottom = 0<br>
pad_left = 0<br>
pad_right = 1</p>
<p dir="auto">How tensorflow do a convolution with a kernel of height 1 on a image of height 5 and which gives output of height 1 (stride = 5) ??? How TF do this ? The doc can't explain the method used ...</p>
<p dir="auto">Doing retro engineering, I saw that TF apply the filter on the middle of the input tensor (pad_top = -2 and pad_bottom=-2).<br>
I agree with this method, but the formulas of the Convolution doc is doing max(.., 0) so padding could never be negative (according to the doc).</p>
<p dir="auto">Could someone explain me clearly what is the formula used in tensorflow ?<br>
Could someone update the doc ?</p>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code -- YES, only to demonstrate the problem (source code is below)</strong>:</li>
<li><strong>OS Platform and Distribution (Linux Ubuntu 16.04)</strong>:</li>
<li><strong>TensorFlow installed from (binary (PIP))</strong>:</li>
<li><strong>TensorFlow version (1.4.0)</strong>:</li>
<li><strong>Python version (2.7.12)</strong>:</li>
<li><strong>Bazel version (N/A)</strong>:</li>
<li><strong>GCC/Compiler version (N/A)</strong>:</li>
<li><strong>CUDA/cuDNN version (N/A)</strong>:</li>
<li><strong>GPU model and memory (N/A -- CPU only)</strong>:</li>
<li><strong>Exact command to reproduce (See Source Code Below)</strong>:</li>
</ul>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">There is an inconsistency between the convolution documentation on padding with 'SAME' located <a href="https://www.tensorflow.org/api_guides/python/nn#Convolution" rel="nofollow">here</a> and the behavior of the tf.nn.conv2d operator. In the example below I create a 3x1 input with values [[1.0][1.1][1.2]] and a 1x1 filter of value [1.0]. I specify the stride to be 1x3x1x1 which should result in only a single element be output and the padding to be 'SAME'. From the padding calculation in the above link:</p>
<p dir="auto">pad_along_height:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="in_height ( = 3) % strides[1]( = 3) == 0 so
pad_along_height = max(filter_height ( = 1) - strides[1] ( = 3), 0)
pad_along_height = max(-2, 0) = 0"><pre class="notranslate"><code class="notranslate">in_height ( = 3) % strides[1]( = 3) == 0 so
pad_along_height = max(filter_height ( = 1) - strides[1] ( = 3), 0)
pad_along_height = max(-2, 0) = 0
</code></pre></div>
<p dir="auto">pad along_width:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="in_width ( = 1) % strides[2] ( = 1) == 0 so
pad_along_width = max(filter_width( = 1) - strides[2] ( = 1), 0
pad_along_width = max(0,0) = 0"><pre class="notranslate"><code class="notranslate">in_width ( = 1) % strides[2] ( = 1) == 0 so
pad_along_width = max(filter_width( = 1) - strides[2] ( = 1), 0
pad_along_width = max(0,0) = 0
</code></pre></div>
<p dir="auto">My hypothesis is that pad_along_* is not using the max(x,0) and as a result, pad_along_height = -2. Therefore pad_top = -1 and pad_bottom = -1. If that was the case, then our input is reduced to only the middle element [1.1] which explains why the TF result of the code below is 1.1 rather than the expected 1.0 (value of first input).</p>
<p dir="auto">If I change the padding to be VALID (no padding) then this code below gives the result of 1.0 or if i instead change the stride to 1,2,1,1 i get the expected value of 1.0 (although in this case my hypothesis proposes that pad_bottom is still -1).</p>
<h3 dir="auto">Source code / logs</h3>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
import numpy as np

i = tf.constant((np.ones(3) + np.arange(3) * 0.1).reshape(1,3,1,1), dtype=tf.float32, name='input')
f = tf.constant(np.ones(1).reshape(1,1,1,1), dtype=tf.float32, name='filter')

conv = tf.nn.conv2d(input=i, filter=f, strides=(1,3,1,1), padding='SAME')

with tf.Session() as sess:
    out = sess.run(conv)
    print out"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
import numpy as np

i = tf.constant((np.ones(3) + np.arange(3) * 0.1).reshape(1,3,1,1), dtype=tf.float32, name='input')
f = tf.constant(np.ones(1).reshape(1,1,1,1), dtype=tf.float32, name='filter')

conv = tf.nn.conv2d(input=i, filter=f, strides=(1,3,1,1), padding='SAME')

with tf.Session() as sess:
    out = sess.run(conv)
    print out
</code></pre></div>
<p dir="auto">Output:<br>
<code class="notranslate">[[[[ 1.10000002]]]]</code></p>