reducing(or stopping)the pop when a speaker is switched
Use the MUTE or SHDN inputs of the PAM8403 (pin 5 or 12) with your arduino instead of switching with a MOSFET. Popping sound will be greatly reduced. Also, put an high pass filter on the inputs as shown in the datasheet. Its cutoff frequency should match your speakers' bandwidth.