title: Deep neural network skip connection implemented as summation vs concatenation?
body: <p>In deep neural network, we can implement the skip connections to help:</p>
  
  <ul>
  <li><p>Solve problem of vanishing gradient, training faster</p></li>
  <li><p>The network learns a combination of low level and high level features</p></li>
  <li><p>Recover info loss during downsampling like max pooling.</p></li>
  </ul>
  
  <p><a href="https://medium.com/@mikeliao/deep-layer-aggregation-combining-layers-in-nn-architectures-2744d29cab8" rel="noreferrer">https://medium.com/@mikeliao/deep-layer-aggregation-combining-layers-in-nn-architectures-2744d29cab8</a></p>
  
  <p>However, i read some source code, some implemented skip connections as concatenation, some as summation. So my question is what are the benefits of each of these implementations?</p>