Why do we always use square kernels for filters?
If the kernel is odd then you can center it on top of a pixel, which is nice. Lots of kernels are symmetric, so that's probably why they are often square. You can certainly use non-square or even kernels though.