Switch 5V with back voltage protection
Basic P-MOS switching circuit:




  The problem is that the body diode of the FET is going to conduct if an external power source is present. 


You can connect two FETs in series with their body diodes opposite each other so they don't conduct when off, and get low drop-out when both are on.  Here's a crappy illustration of the concept from a product: