How to get rid of low quality sound out from DFPlayer plus amplifier?
The DFPlayer already has a 3W amplifier on it, so you can connect a speaker directly to the DFPlayer. There is no need for an extra amplifier.
If you do connect an extra amplifier to the DPlayer, and turn the volume of the DFPlayer up too far, the input of the second amplifier will clip and it won't sound good. Also, the output of the DFPlayer is probably from a class-D amplifier designed to be connected directly to a speaker; it won't play nice with other amplifiers.
You should try how the DFPlayer sounds when you connect a speaker directly to it. The 0.5W speaker you have is not a good fit; it can't handle the power of the DFPlayer's amplifier.
A 3W (or more), 8Ω speaker would be good for this if you want to use the maximum power the DFPlayer can deliver. According to most publications on the interwebs a 2W, 8Ω speaker will also do, but I would be careful with turning the volume all the way up.
Note that the DFPlayer can pull a fair bit of current (about 25mA in standby, and in the order of 200mA when playing) and it needs a power supply that can deliver that.