Which (video game) recording program fits for these criteria?
None.  If you want low CPU/GPU usage and small file size, you need a hardware encoder.  Period, end of story.  Games are CPU/GPU intensive and video compression is also exceedingly demanding on CPU/GPU.  Programs like FRAPS capture video smoothly by postponing the compression until later and instead save video as very large, minimally compressed files.  This works because most games do not make intensive use of the disk drives.

You can either capture in large files with minimal impact and then batch transcode them to a smaller size or you can buy a dedicated hardware encoder that can take the load off the CPU and GPU. Those are your only two options.

Update: If you only need 10 hours to fit within 100GB, then you might be ok for an SD feed.  It doesn't really have to be that highly compressed at that low of resolution.  It is still going to hit your CPU or GPU though, so gaming performance will be impacted, just not quite as badly as trying to do something like h.264.

I'd probably start by trying something like a DV codec and see how your computer handles it.  Millage will vary based on how much your computer actually needs for the game.  Hardware encoders or software encoding after the fact are still the best option overall though, since they maximize the quality of the recording.  (Fast compression is not good compression unless it is specialized hardware.)