title: Stuck understanding ResNet's Identity block and Convolutional blocks
body: <p>I'm learning Residual Networks (ResNet50) from Andrew Ng coursera lectures. I understand that one of the main reasons why ResNets work is that they can learn identity function and that's why adding more and more layers in network does not hurt the performance of the network. </p>
  
  <p>Now as described in lectures, there are two type of blocks are used in ResNets: 1) Identity block and Convolutional block. </p>
  
  <p>Identity Block is used when there is no change in input and output dimensions. Convolutional block is almost same as identity block but there is a convolutional layer in <code>short-cut</code> path to just change the dimension such that the dimension of input and output matches. </p>
  
  <p>Here is identity block:</p>
  
  <p><a href="https://i.stack.imgur.com/37qzA.png" rel="noreferrer"><img src="https://i.stack.imgur.com/37qzA.png" alt="enter image description here"></a></p>
  
  <p>and here is convolutional block:</p>
  
  <p><a href="https://i.stack.imgur.com/0mE2p.png" rel="noreferrer"><img src="https://i.stack.imgur.com/0mE2p.png" alt="enter image description here"></a></p>
  
  <p>Now in implementation of convolutional block (2nd image), First block (i.e. <code>conv2d --&gt; BatchNorm --&gt; ReLu</code> is implemented with <code>1x1</code> convolution and stride > 1.</p>
  
  <pre><code># First component of main path 
  X = Conv2D(F1, (1, 1), strides = (s,s), name = conv_name_base + '2a', padding = 'valid', kernel_initializer = glorot_uniform(seed=0))(X)
  X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
  X = Activation('relu')(X)
  </code></pre>
  
  <p>I don't understand the reason behind keeping stride > 1 with window size 1. Isn't it just data loss? We are just considering alternate pixels in this case. </p>
  
  <p>What should be the possible reason for such hyperparameter selection? Any intuitive explanation will help! Thanks. </p>