PDM to PCM conversion workload on 32-bit micro (such as STM32)
Have you considered the analog option? Just add an RC filter to the output of the MEMS microphone, connect the filter output to the built-in ADC, and let the analog electronics do the filtering for you. A ~4 kHz cut-off frequency (say, 3.9kΩ R plus 100 nF C) would give you telephone-quality audio, and should be pretty cheap to build, with a small board footprint.