How can I reduce the power consumed with wall warts?
I would go with a single mains-to-12V adapater, preferably one that's a switching model.

The switcher will waste less energy generating the +12V rail than the linear, and can regulate its output without dummy loads / burning power unnecessarily.

For the 5V rail, consider building a 12V-to-5V buck converter if the load current justifies it.

("Less than 0.5A" makes the decision less than clear - how much less than 0.5A? If it's 100 microamps, use a linear regulator. If it's 0.49A, you're probably better off with a buck.)