What happens to unused power in generators
There are two aspects to control of the generator- the first is the motor RPM and that will determine the AC output frequency. That is controlled (typically mechanically with a governor) by adjusting the throttle automatically (you may be able to see the throttle linkage). That maintains fairly constant RPM as the engine loading changes. 

The second is voltage regulation and that is sometimes achieved by changing the field coil current (inexpensive or small ones may have fixed excitation so the voltage will drop noticeably as the load increases). 

At a very light (electrical) load, the generator is very easy to spin so the motor is essentially idling. As you add electrical load the field caused by the current flowing through the load (and the coils in the generator) starts to oppose the rotation of the rotor (Lenz's law) and it becomes much harder to spin the generator. (The voltage will also start to drop so field current also increases to maintain the voltage if the generator has an AVR- which will also increase the required torque to compensate for the internal losses). 

As the load increases the motor RPM will tend to drop, so the governor opens the throttle wider, admitting more fuel and air to the engine, in order to try to maintain a constant RPM. It's probably a reasonable rough approximation to say that the fuel consumption increases linearly with kW loading above the baseline idle consumption. 

TL;DR: Yes, it will suck a fair bit more fuel if you put a heavy load on the generator.