Put a resistor in series between amp and speaker
Cheap class D amplifiers have very poor rejection of power supply noise/ripple. This is fully expected because the output square wave peak IS the power rail so, the trick is to make that power rail as quiet as possible. So, if your output signal is quite small, noise/ripple on the power rails can dominate.


  This 10k resistor seems (by ear) to make something like -20dB volume
  reduction. Is there a formula to find R for -10dB reduction?


A 20 dB volume reduction is about a 4:1 decrease in sound level as perceived by the ear because 10 dB (or 1 bel) is "defined" as the attenuation that halves loudness at 1 kHz. This doesn't hold true for other audio frequencies but it's a good measure.

If your loudness has reduced by 4 then that is an attenuation of 20 dB. If your earphones have an impedance of about 1000 ohm then adding a 10 k resistor will attenuate the signal to 1000/11000 = 0.09 and if you take the log and multiply by 20 you get 20.82 dB.