Bypass capacitor theory
A typical opamp used in audio has a gain bandwidth product somewhere in the one to a few tens of MHz, so the things usually have quite some gain outside the audio band. 

Also, it is instructive to remember the current flows in loops, and that typically the opamps output is referenced to some 'ground return' which is NOT directly connected to one of the opamps power pins (They are often +-15V or so), the bypass caps near the opamp serve to close this loop and minimise the potential for magnetic coupling (proportional to loop area). 

A further bit of nastyness is that a class AB opamp will impose half wave rectified current pulses onto the tow power rails as the output switches from sourcing to sinking and vice versa, the local caps help minimise these loop areas too. 

Now 100nF was sort of standard for ceramics, back in the day but these days 1uF is readily available in 0603 X7R and is worth using, and wise designers tend to use a local electrolytic as well, a few 10s of uF is useful, do not go for overly low ESR here. Some of us like a modest series resistor before this arrangement as well, it both adds a pole and helps to damp any resonances between the power network and the rather large caps back at the board entry, 10 ohms or so is usually ok. 

It is surprising how little series inductance it takes when you have 1000uF across the input to give a resonance in the audio or low ultrasonic region.

I only really got this stuff once I stopped thinking about voltage as somehow being the primary thing and started thinking about current (and its loops) as being at least as big a consideration, for all that the schools seem to concentrate of voltages as being the thing.