How does one get started with electrical engineering?
Buy a friendly, practical, basic general electronics book. One of the best I've seen for a complete beginner is "Practical Electronics for Inventors" (a 3rd edition has just come out) It covers all the basic theory and components and their uses has a chapter on common electronics design areas (audio, optoelectronics, digital stuff, etc)
Seen by many as the "bible", and probably required reading at some point is "The Art of Electronics". Quite old now (everyone is waiting for a new edition) but the detailed treatment of basic analog electronics (transistors/opamps/filters/regulators/etc) is still just as important. More good books recommended here.
Buy/assemble a basic electronics kit to experiment with. There are plenty of circuits in the above mentioned books you will want to try out. This should include a breadboard, some batteries/small power supply, soldering iron/solder, common values of resistors/capacitors/inductors/potentiometers/crystals, common transistors, common opamps, LEDs, 555 timer, a few different logic gates and maybe a few sensors (LDR, electret microphone, pressure sensor, temp sensor) and a DC motor and/or stepper motor.
Anyway, here are couple of lists/kits:  

Starter kit list
Various kits of common components
Basic starter component kit
Starter electronics experimenter kit (pictured below)


Spend lots of happy time reading, trying stuff out and making the odd bit of magic smoke :-)  
Maybe move on to a microcontroller kit if that's the way you want to go (almost all electronics nowadays involve a micro so it's a good idea at some point)
Keep it simple to start with though, if you are serious I would maybe avoid Arduino and start with a simple standard microcontroller kit (e.g. PIC16F/18F(Starter Kit)/32F, Atmel AVR, TI MSP430), but if you want an easy/friendly/well supported platform then go for it. Note that you will need to learn how to program on a memory limited embedded device, which will (unless you want to stick to assembler only) almost certainly involve learning C (and a bit of assembler is a good idea)
Here is another PIC starter kit: