Volume control for LM386
You probably used a linear (B) taper pot. For audio you want an (A) log taper pot, so your 500 ohms will be about 25% of full electrical rotation, not 5%. 

Image from this site.