Frequency filtering with a DFT and meaning of removing complex conjugates
Here is a different way to look at it:

The violation of complex conjugate symmetry produces and "error". That error is, that the impulse response becomes complex. By applying the "real()" operator on the inverse FFT, the error is eliminated again.