What causes periodic noise with LM386 amplifier
Adding a 100uF decoupling capacitor as suggested in the comments significantly reduced the noise. I also found that the noise was being induced by a MCU that was blinking a led and was powered by the same battery.