Why do speakers sound different when they're not fully plugged in?
The simple answer is:  Changes in resistance/impedance at the connector or cable can change the actual or perceived frequency response that you hear from the speaker.  

In the case of the actual frequency response:  The connector and stuff inside the speaker form an RC filter (a low-pass filter).  The connector is the R and the stuff inside the speaker is the C.  As R goes up, the cutoff frequency of the low-pass filter goes up.  Normally, the R is less than 1 ohm, so the cutoff frequency is very low (less than 5 Hz).  But if the connector is bad or dirty then the R could be several hundred ohms, which could increase the cutoff frequency by several hundred times.  The net effect is that the bass is all cut out.

And for perceived frequency response:  Human hearing is very weird.  At low volume levels, our ears simply don't pick up some frequencies very well-- usually the low and high frequencies.  We're more sensitive to mid frequencies.  So, as you reduce the volume level we perceive that the low and high frequencies are being cut out more than the middle frequencies.  The "Loudness" button of a stereo attempts to compensate for this by boosting those frequencies and therefore making us hear a more balanced sound at low listening levels.  If the connector is bad or dirty, less power will be making it to the speaker and causing a lower volume level.  But due to human hearing, we would also hear less low and high frequencies-- making it sound like we're listening through a phone or something.

This is an over-simplified answer, but reasonably accurate for your purposes.