Why CNN input images are often square shaped?
I'm not sure, (also studied the CS231n class), but I'm guessing it's something to do with:


matrix operations, 
convention, and/or 
control size of image out.


Not a lot of evidence points towards (1), cause we have insane libraries to do this (ATLAS, MKL, or OpenBLAS). (2) makes more sense practically. (3), because you mentioned zero padding (hyperparameter), the CS231n course says this is done to control the size of output from the current CNN layer and control the spatial size so that input (h x w) = output (h x w). Also, I guess it also reduces errors in matrix operations when you have the same image size (h = w) instead of (h > w or h &lt; w).

Most likely, (3) is the predominant reason. This is just my thoughts. Someone please confirm.