N MOSFET Issue in Battery Charging Circuit
Let's start with what appears to be the obvious - you have drawn the mosfet symbol and you have indicated the direction of the parasitic diode. Now what does that diode tell you about the direction of current flow when the USB is removed - it should tell you that your FET is either the wrong way round or not suitable for this type of circuit because it will discharge the LiPo into the LED when the USB is not plugged in.

The simplest way to charge the LiPo is to use a schottky diode from the 5V with cathode to the LiPo. This cannot feed current back to the LED when the USB is removed.

However, the problem you now have is that there is a small diode drop but, from what I know about LiPo terminal voltage (which isn't much) is that they operate somewhere less than 4.5V so maybe it will charge OK with the inbuilt protection.

Maybe you could post a link to the battery you are using so this can be confirmed.

More detail about why the FET drawn is inappropriate It's an N channel FET which is fine for switching loads and batteries but it would switch on to low values of on-resistance in the negative terminal of the LiPo because you could raise the gate to the positive LiPo terminal and turn on the FET properly. However, the parasitic diode would still be forward biased when the USB voltage is removed so you can't win on this with a simple FET circuit methinks.