Jumper out an LM7805
Well, I decided to simply breadboard it. See below for the Summary.


  I fully recognise @glen_geek's comment regarding overlooking the jumper - but I figure that's my lookout!
  Yes, I realise that this is a single experiment and not to trust this empirical result. However...


Experiment

For those who are thinking of doing the same, I implemented the following schematic:



TP1: Current measurement (mA)
TP2: Voltage measurement (V)
LM7805: Regulator temperature (Tr)
20R 5W: Dissipation temperature (Td)
JP1: When Batt was 9V, the jumper was OFF.
     When Batt was 4.5V, the jumper was ON.


1) 9V with JP1 OFF:
   mA = 250 mA
   V  = 4.99V
   Td = 36C
   Tr = 39C

2) 4.5V with JP1 ON:
   mA = 200 mA, descending to 180mA over 1 hour
   V  = 4.08V, descending to 3.51V over 1 hour
   Td = 36C
   Tr = 22C (ambient)

3) Back to 9V with JP1 OFF:
   mA = 250 mA
   V  = 4.95V
   Td = 36C
   Tr = 39C

4) 4.5V with LM7805 removed! (JP1 ON):
   mA = 180 mA
   V  = 3.52V (same old batteries as before)
   Td = 36C
   Tr = N/A

Summary

In short, with a 200-250mA load:


The regulator got hot when I expected it to;
The regulator was cold when it was jumpered over;
The source current didn't change regardless of voltage supply;
The regulator did its job again after being abused for an hour;
Removing the regulator did nothing for the low-voltage scenario.


I'm going to risk it!