Adding a third Airport Express causes dramatic network slow down
This sounds as a channel overlap problem to me. Apple Airport devices try to use distinct and far-apart channels but may make a wrong automatic choice when there is a lot of interference from other WiFi stations, sometimes depending on the order you turn them on.

If you really need four access points (Extreme + 3 x Express) you either live in a really big house or those APs are not optimally placed. I'd suggest first using one AP (the Extreme), than go around every room and plot your signal strength and take note of other networks that pop up with their channel info. That way you find dark spots in the house and strategically place the Expresses to cover them.

Additionally, if you only need the AE to Airplay music to the speakers on the audio port you actually don't need to create an additional wireless network.