Li-Poly battery charger with protection - Schematic review
Your design will trip the PTC polyfuse in your USB port when the load demands 500mA and the battery demands anything significant. Since USB2 only supports 500mA max. USB3 supports 900mA.  

It is not clear to me how your device load is switched on/off so you need a smart load sense switch to U1 CHRG using a comparator.

But it is clear to me this approach will not work. YOu want a trickle charge when your device is off, so the USB can power the unit and the external Li-Po can get a trickle charge, If I understand you are saying and then when the USB is off but your "load is still on" it can run off the Li-po as long as it is charged.

What you could do is .. add a small shunt resistor between USB (Vbus) and LOAD (VSWITCH) with a comparator to disable U1 CHRG when there is current being used.
.