How much, generally speaking, does it cost to learn how to fly?
Assuming a regular private pilot's license (i.e. not sport/recreation), a part 61 instructor, and a rented aircraft, then you need to budget for the following:


Aircraft rental (this usually includes oil and fuel)
Instructor time (air and ground)
Materials (books, DVDs, charts, fuel tester etc.) as recommended by your instructor
Headset
Third-class medical
Insurance
Written test fee
Examiner's fee for the actual test


If we assume the aircraft costs \$120/hr wet (i.e. including oil and fuel), an instructor costs \$40/hr and you need 50 hours to be ready for the test (40 is the minimum required) then you can do a calculation like this:


35 flying hours with instructor: \$5600
15 flying hours solo: \$1800
15 hours ground school (instructor's time): \$600
Materials: \$500
Headset: \$500
Third-class medical: \$150
Renter's insurance: \$700 (per year)
Written test fee: \$150
DPE test fee: \$400 (plus \$120 rental for the aircraft)
Total: \$10,520


You can play around with these numbers, there's a lot of variation and research is obviously important. But the single largest cost will be the number of flying hours you need, so anything you can do to reduce that time is worthwhile and the simplest - but unfortunately not the easiest - way to do it is to fly as much as possible, i.e. several times a week. Given weather and other factors, you should probably even plan to fly more than you actually can on paper: a bunch of flights will certainly be cancelled.

You can also train in the most 'minimal' aircraft you can find: a two-seater aircraft without a glass panel could be \$50/hr cheaper than a four-seater with a G1000. Unless you have a very clear idea about what sort of flying you'll do in the longer term there's no point in paying for anything that isn't directly required for the private pilot's test.