How to attenuate a cheap MP3-player speaker?
For this simple thing, where audio quality probably isn't very noticeable, just put an inline resistor in series with one of the speaker wires.
A 4ohm should cut the sound roughly in half.  But depending on your preference you may want a larger value. 
Make sure the resistor power is large enough for that amp.  It says it's a 3W, but I doubt that it is being used at that level.  Since the power will be split between the resistor and the speaker, you can probably use a 1/2W.

Sounds like a fun experiment, play and have fun!