DFT and Inverse DFT in Image Processing
I saw same issue, and I found the answer here.
for short, you can set imaginary part of filtering kernel zeros before you call mulSpectrums.  

Mat planes[] = {Mat::zeros(dftImage1.size(), CV_32F), Mat::zeros(dftImage1.size(), CV_32F)};   
Mat kernel_spec;   
planes[0] = your_filter; // real     
// planes[1] = your_filter; 
// imaginar should be zero! Don't touch it.   
merge(planes, 2, kernel_spec);     
mulSpectrums(...  


Hope this helps.