Is it a good idea to use different width and height of the kernel in a CNN?
It depends on your application. In case of text recognition, non-uniform kernels are used since the information about text is less on the horizontal axis and more on the vertical axis.
If in your case it is applicable then, it will be good idea. But, if it is not you are better off using a smaller uniform kernel (2x2, maybe). You can also zero-pad your image to make it uniform before putting it through convolutions. Also, check if you are doing 'valid' or 'same' padding in your convolutions since 'valid' convolutions chip away at your image dimensions.