Most efficient way to improve my house's heating
Replacing the windows with modern double-paned isn't worth the cost; they're not that much better. However, I would recommend looking around for gaps around the windows where the putty stuff has dried out and repairing them. That's a mostly-labor job, and isn't that hard.

In general, your first goal should be to seal any gaps. This is not only much cheaper than buying insulation, but can be done DIY. Look around windows, doors, and vents, as well as any other place where there are intentional holes in the house. These tend not to be done that well, and you may find unintentional gaps.

Is your house on a slab? You didn't talk about the floor insulation.

A new boiler could directly save you roughly 10% on your gas bill, but would be expensive. (You can run the numbers yourself.)