Remapping Component Selection Hotkeys
You can use Ctrl+Tab and select Vertex, Edge or Face from the popup menu, which will appear under the mouse. As far as I know, this is the fastest way to switch.

Update for 2.8: This function is now available by default when pressing 1, 2 and 3. To remap this back to the original behavior see this question.