What good resources are there for a high school student to start learning electronics?
Considering you're of 'high school age', I think the MIT courses are maybe very difficult (from Mark's post) They generally require a lot of math and previous knowhow from highschool. Or am I only able to pick out the courses that handle circuit theory and maths?

You may find this very interesting:

http://www.allaboutcircuits.com/vol_1/index.html

Start at chapter 1 and read through. You may find some parts that you think is familiar, but read through it anyway. You may already know what a Volt and an Amp is, but it might also be nice to read through some literature so you can conform your previous knowledge. 

This website is focussed on analog electronics. It's about understanding how electricity and electronic components work. This doesn't have a particular big focus on digital electronics, which you may want to know more about considering you're interested in writing computer programs.

As Mark said, Arduino is a great platform to start from for digital electronics. It contains a small microcontroller with an easy to pick up language to start with programming embedded systems. Out of the box you won't be that concerned with configuring registers of microcontrollers. You can get a ton of addons created by 'the community' for it. You can get a ton of stuff from Sparkfun.