title: Does it make sense to use skip connections in networks with less than ten layers?
body: <p>I was reading about the skip layer connections in the ResNet paper, and how they were beneficial in training very deep networks. </p>
  
  <p>Does it make sense to use such connections in smaller (i.e. AlexNet-like) networks of less than 10 layers? </p>