Using a capacitor to properly power a servo
"Why was a capacitor necessary to "stabilize" my Arduino?" 


A servo motor draws a substantial current in short peaks. And USB power is not designed to provide such current peaks (and the cable makes it worse), hence the current peaks cause the voltage to vary, which the Arduino is not designed for. A capacitor acts as a buffer for current, so as long as the USB power can deliver the average current, the capacitor will help by smooth out the current peaks.


"Why parallel and not series?"


Think of a capacitor as a reservoir. But beware of overinterpreting anlogies, better read a decent electronics beginners book.


"What's so special about a 470uF capacitor (vs., say, 100uF)?"


The uF figure is the amount of buffering. Enough is enough, but better be on the safe side (for this use, more is OK). The value of 470 uF is probably based on experience. Just by accident, it is the same value I let my students use :)


"I'm now afraid to even touch the capacitor on my breadboard!"


Capacitors can be dangerous if they store either a high voltage or a large amount of energy that can be released in a very short time. In your case neither applies, so don't worry. But you might see a small spark when you short the leads of your capacitor, even after you have removed the USB power. A resistor (for instance 1k) in parallel to the capacitor solves this.