MP3 player shield makes noise
You likely have a ground loop, between the chips GBUF and the speakers GND, caused by the speakers power supply. Note the SFE MP3 player is like your referenced vs10XXan_output.pdf Section 2.1 Figure 4 or 5 (minus the ESD). 



Note the GBUF (the return of the headset) is not GND. The VS1053's output is likely a Class D amplifier. Hence it's switching at high frequency, in reference to the return (or GBUF). Section 3.2 recommends that when the VS1053 is driving a amplifier, it requires the addition of C1,R1,C2,R2 as shown in figure 8.. The SFE MP3's schematic does not have these. leading to some complaints, such as your.



The SFE MP3 and UNO with a no-name amplified speaker set works for me. But I believe my speakers have AC isolation within, as it only has hot and neutrual and no ground, yours likely has some connection to ground. Hence you are hearing the switching across GBUF. Additionally there are warnings stating this can be harmful to the amplifier.

Until SFE changes their design, the only solution is either a isolated amplifier or hack a cable to include the addition AC coupling (aka R's and C's).

Regardless the VS1053 and its open source Library are a great companion for the Arduino. I have seen many successful projects using it.