Convolution theorem with a kernel smaller than the image
If you want to do the convolution in the frequency domain, you have to pad the kernel with zeros to make it the same size as the image.  There is really no way around it.  The upside is that you only need to take the FFT of the filter once, and then you can re-use it for multiple images.

If you are concerned with speed, you would have to do some analysis here. For small separable kernels doing the convolution in the spatial domain may be faster.