supply two board with one battery
When you use different source for your project then you isolate both module from each other 'noise', or in this case the change of voltage caused by the osciliation of audio signal. Then when you connect the two source together, then both would share the noise (I mean the audio signal alone probably change the supply voltage, you can check this with multimeter/osciloscope when you use different supply). Then when you connect both, its either the change messing with the voltage which should be stable, or the stiff supply makes the audio signal's information change.

My suggestion is to isolate the two modules source, by using voltage regulator or just simple L-C filter shhould work.

Hope it helps, Cheers