title: Effect of max_pool in Convolutional Neural Network [tensorflow]
body: <p>I'm following Udacity Deep Learning video by Vincent Vanhoucke and trying to understand the (practical or intuitive or obvious) effect of max pooling.</p>
  
  <p>Let's say my current model (without pooling) uses convolutions with stride 2 to reduce the dimensionality. </p>
  
  <pre><code>  def model(data):
      conv = tf.nn.conv2d(data, layer1_weights, [1, 2, 2, 1], padding='SAME')
      hidden = tf.nn.relu(conv + layer1_biases)
      conv = tf.nn.conv2d(hidden, layer2_weights, [1, 2, 2, 1], padding='SAME')
      hidden = tf.nn.relu(conv + layer2_biases)
      shape = hidden.get_shape().as_list()
      reshape = tf.reshape(hidden, [shape[0], shape[1] * shape[2] * shape[3]])
      hidden = tf.nn.relu(tf.matmul(reshape, layer3_weights) + layer3_biases)
      return tf.matmul(hidden, layer4_weights) + layer4_biases
  </code></pre>
  
  <p>Now I introduced pooling: Replace the strides by a max pooling operation (nn.max_pool()) of stride 2 and kernel size 2.</p>
  
  <pre><code>  def model(data):
      conv1 = tf.nn.conv2d(data, layer1_weights, [1, 1, 1, 1], padding='SAME')
      bias1 = tf.nn.relu(conv1 + layer1_biases)
      pool1 = tf.nn.max_pool(bias1, [1, 2, 2, 1], [1, 2, 2, 1], padding='SAME')
      conv2 = tf.nn.conv2d(pool1, layer2_weights, [1, 1, 1, 1], padding='SAME')
      bias2 = tf.nn.relu(conv2 + layer2_biases)
      pool2 = tf.nn.max_pool(bias2, [1, 2, 2, 1], [1, 2, 2, 1], padding='SAME')
      shape = pool2.get_shape().as_list()
      reshape = tf.reshape(pool2, [shape[0], shape[1] * shape[2] * shape[3]])
      hidden = tf.nn.relu(tf.matmul(reshape, layer3_weights) + layer3_biases)
      return tf.matmul(hidden, layer4_weights) + layer4_biases
  </code></pre>
  
  <p>What would be the compelling reason that we use the later model instead of no-pool model, besides the improved accuracy? Would love to have some insights from people who have already used cnn many times!</p>