Is it safe to bypass an LDO by shorting its input and output?
As far as I know this is actually a fairly common solution. I know I've used this setup in the past without any problems.   

This application note from LT even shows how to do this automatically, using a FET as the bridge: 5V to 3.3V Regulator with Fail-Safe Switchover – Design Note 82.