Is there a way to step up voltage without consuming power when there's no load?
Any circuit that you put across the battery will draw some power, but with no load it can be very low. The problem comes when a load is applied and it needs to hold the voltage up. The more power it has to provide under load, the higher the quiescent current may have to be for it to work properly.

The 'thing' you are talking about is called a Voltage Booster or DC/DC Step Up Converter. The two most common ways to step up a DC voltage are:- 


Switched Capacitor 


A capacitor is connected in parallel with the battery to charge it, then reconnected in series to double the output voltage. With no load the capacitor will stay charged for a long time, but under load it will start to discharge and so must be topped up periodically.

The switching circuitry will draw a bit of current even when there is no load. This quiescent current can be reduced by lowering the switching frequency, but then the output voltage will drop more as loading increases. 


Inductive booster


An inductor (coil of wire) is switched across the battery, causing an increasing current flow and building up a magnetic field in it. Then the switch is opened, and as the inductor's magnetic field collapses it tries to maintain the current flow - increasing voltage until it does. A diode steers this voltage to the output, where it charges up a capacitor. 

An inductive booster must draw some current from the battery even with no load. However it may be able work in 'discontinuous' mode where it draws a short burst of current to charge the output capacitor, then waits for the voltage to drop a little. If capacitor leakage is negligible then the duty cycle (ratio of switch on to off time) can be very low, resulting in very little average current draw from the battery. Quiescent current draw will then be determined by how much power the circuit needs to work in this mode.