Use battery to drive main circuit or load battery and cut-off main circuit depending on power source
From what I understand, your device either runs from battery or, if it's charging, your device is then unpowered and only charging battery. So you could have a P-Ch MOSFET on the main power line that will be open when Vcharger=0V and closed when Vcharger=5V. I suggest you connect Vcharger to the gate of that MOSFET (via some 100R) and have a pull-down of some 10k on it too. So if there is no charger, your MOSFET has gate to the ground and it's conducting, but when you plug in the charger, it closes.
Here is my crude 1 min paint ullustration. Let me know if I understood you wrong (I skipped the charger circuit itself, the one the converts 5V into some 4.2V, it doesn't matter here, it's obviously between the battery and the charger, here the focus is on MOSFET gate):