Impedance matching between CSR8675 and a MM20 microphone
The advantage of impedance matching is that you have maximum power transfer. The advantage of voltage matching is that you have no signal amplitude loss under different load conditions (e.g. cable length or multiple loads). 

I think with the setup you described, it does not matter what you do.