Why would an audio circuit output through two caps like this?
The components used are electrolytic capacitors. These components are unipolar or polarised, meaning that the voltage over them can't simply be reversed without the risk of destruction and even explosion if the source can supply enough current. 

Sound is AC. So if you want to use an electrolytic capacitor to decouple successive amplifier stages you must bias it in the correct way with a DC source to prevent the AC from reversing the polarity, like is done here. 

 

If biasing isn't an option an alternative is to put 2 capacitors with the opposite polarity in series. That creates a bipolar capacitor. The downside is that the capacitors need to have twice the capacity to produce the same value a single polarised device.