Introducing hobby projects
I am in the same position, and I chose the Arduino.  There's a nice software development platform, lots of instructional materials, and numerous vendors to choose from.

There's even a really nice open-source "getting started" tutorial.

Check out this beginner's kit.  I bet you can adapt this with the clones mentioned by RedGrittyBrick and have a very nice, low-cost setup.  Let us know what you end up doing!

Sparkfun Inventors Kit
http://www.sparkfun.com/products/10173