What initial improvements can I make to a Schwinn Sidewinder?
$50 - $100 isn't going to get you much in parts, especially any that would be an "upgrade" from your current setup. If you're riding your bike often, it's possible that you'll spend an amount approaching that this year on new tubes and/or tires when you get a flat or wear your tires out.

My suggestion would be to ride this bike and enjoy it. You'll get your exercise and it'll give you time to research bikes and save for one that will fuel your desire for more advanced riding down the road when you gain your fitness. Visit some bike shops in your area and talk with the staff there to find a bike that is in your range of needs and budget. Test ride lots of bikes to find one that fits properly.

You don't need the "fastest" or most modern/expensive bike if you just want exercise. You just need something that is safe and functions for your needs. The best upgrade you can give your bike is the motor (yourself).