Removing noise induced by IC (Ground loop?)
I see several issues that need resolving:

The capacitor has a rating of 6.3V which is lower than your supply voltage of 7.4V
The added inductance of the wiring to your capacitor might cause it to resonate
The connection between the grounds does not seem solid
For a class D amplifier there seems to be a lack of big decoupling capacitors on the amplifier PCB

Try the following:

Use a capacitor with a higher voltage rating; preferably 12V or higher. Your current capacitor could be damaged and show unwanted behaviour.
Ensure that the capacitor is placed as close as possible to the power supply pins of the amplifier (least amount of wire in between). This will reduce the area that the current travels and with it the voltage peaks.
I would remove the inductor; should the mp3 decoder suddenly peak or drop in its current demand than the inductor could create a voltage drop or spike.
Insert the ground wires into the holes opposing the copper pads and solder them. Take a clipped leg from a through hole component or some bare wire and lay it over the copper pads of all the ground wires and solder it. You now should have a decent connection between all the grounds. This prevents high resistance in your ground paths.
Make sure when soldering that you try to get as many wire strands as possible in contact with the to-be-soldered-surface and solder to prevent bad connections.
Try to keep your signal wires away from your 'power' wires as crosstalk can occur between the two.