Arduino continually resets when connected to external power supply, but not USB
Most likely lack of current - lack of current able to be provided by the Mini. The audio is powered from the onboard regulator on the Mini, which has limited current capacity. 

If too much current is drawn it will either get too hot and go into thermal shutdown until it cools down enough, or it will drop the output voltage, making things reset and crash.

Power the audio system from an external 5V regulator.