Help tame turn-on transient
A very simple solution is to put some small amount of resistance or inductance in series with switch (both V+ and V-).  All your 0.1uF caps will have a slightly slower RC charge curve and will dampen the transients.  Of course you need to balance out the size of the resistor with the current and voltage requirements of your amplifiers.  Inductance will choke the transient yet have lower DC resistance.  Also ferrite beads can be used which would probably work the best if you pick one that won't be saturated by your inrush current.

Also don't leave your unused op-amps with their inputs floating.  If conditions get just right they can oscillate and generate lots of noise.