Can a DHT11+ESP8266 be powered by a Adafruit USB LiPo charger directly?
No, you can't.

The charger circuit will only operate properly with a battery attached.

It is not designed to provide power to a system. It is designed for charging the battery and nothing else.  You are supposed to remove the battery from the charger and then attach it to your circuit.  Even with a battery attached you can't use it to power your circuit - the voltage will be too high, and your target circuit will confuse the charger and it will fail to charge the battery properly (or it may end up over-charging the battery).

To do what it sounds like you want to do you need:


A LiPo charge circuit
A 3.3V regulator (ideally a Buck/Boost combined regulator)
Bypass circuitry to power the 3.3V regulator directly from the incoming power when present or from the battery when no external power is present.