Battery Switching Circuit using Transistor and PMOS
Something along the lines of this will eliminate the body diode drop.







simulate this circuit &ndash; Schematic created using CircuitLab

When Control is low M4 is On. The gates of M1/M2 are high and the battery is isolated. 

When The control is high M4 turns off. The gates of M1/M2 are pulled low connecting the battery to external power.

When external power is off R1 pulls the the gates low no matter what the status of the charge control line and the battery is connected to the load.

D1 is required to prevent the battery turning itself off. 
R1 should be small enough that the gates of M1/M2 are discharged sufficiently quickly to avoid an interruption to the loads power.

You could potentially eliminate M2 if you can be 100% certain that external power will always be over the battery voltage when it is applied. M2 ensures that if external power is present and the control line is idle the battery is always isolated even if it's voltage is above that of external power, this is needed if you want to be able to run off a voltage below 4.5V without partially  discharging the battery.

-- Added detail --
This is all assuming that when high the control input is sufficiently close to the external power voltage to turn M4 off fully. (3.3V and 4.5V should be ok assuming a threshold of around -2V which is fairly typical). If however that isn't guaranteed then instead pull the gate of M4 to external power with a 1k and connect the control input to the gate of an n channel. The n channel then connects the gate of M4 to ground when the control line goes high. This inverts the operation of the control input.  It's extra parts and complexity but eliminates the voltage dependency.

This is all assuming you have a smart battery pack with a charger built in and not just a basic protection circuit. Since you refuse to give part details don't blame me if your battery bursts into flames. The external power needs to be around 4.5V so that you get at least 4.2 reaching the battery, I have no idea what headroom the charger needs but it's going to need something over the raw cell voltage in order to fully charge them.