Why CNN input images are often square shaped? Why it's so that in convolutional neural networks we generally take the image dimensions of input image to be generally a square? We even do padding to make it happen. Why not different dimensions?

What I understood is that computer computes multiplication and division (by 2) much faster than the rest.

Can someone shed some light on this? 

Any link or reference will be appreciated. I already have CS231n notes and lecture slides.
I'm not sure, (also studied the CS231n class), but I'm guessing it's something to do with:


matrix operations, 
convention, and/or 
control size of image out.


Not a lot of evidence points towards (1), cause we have insane libraries to do this (ATLAS, MKL, or OpenBLAS). (2) makes more sense practically. (3), because you mentioned zero padding (hyperparameter), the CS231n course says this is done to control the size of output from the current CNN layer and control the spatial size so that input (h x w) = output (h x w). Also, I guess it also reduces errors in matrix operations when you have the same image size (h = w) instead of (h > w or h &lt; w).

Most likely, (3) is the predominant reason. This is just my thoughts. Someone please confirm.