Switch between power supplies with a MOSFET
The problem with your +3V source is that it is ground referenced. 

A MOSFET turns on based on a voltage differential between its Gate and Source pins. If you have access to a 15V supply, that'd turn on the FET if its datasheet calls for 3V Vgs.

You can use a schottky diode  to prevent a voltage drop in the +5V rail.

If you're using a physical switch on the 3V, why don't you just use that switch on the 12V with a schottky?