Voltage supply clipping to <=3.3V
You need a voltage regulator that regulates to 3.3V and as the input voltage drops to 3.3V or below, the output remains close to the input voltage despite it not being able to regulate any more - in other words it acts like a &lt;0.25ohm resistor when unable to regulate.

The LP3964 has a drop-out voltage of 24mV at 80mA and its output will follow the input voltage if the input voltage is too low for regulation at 3.3V. Here is the pdf file for it.

Figure 11 speaks volumes - this is for the 2.5V fixed version but the adjustable-version (set for 3.3V operation) will work just as well: -



I think this nails it really with one exception - it still draws about 3 mA when the voltage is not regulating. Can you live with this? If you can live with the losses of a linear regulator when the battery is at 4.2V and the regulator is producing 3.3V at 50mA (0.9V x 0.05A = 45mW) then 10mW (3.3V x 0.003A) doesn't seem a problem really.