Designing a Li-Ion equivalent USB powered circuit
You are on the right track. The AMS1117 at 3.3 V is perfect for your application. All that your device needs is a voltage in between approx. 3 V (better a bit more) and 4.2 V. You can safely power your system by using the LDO. 

Another story is it if your system wants to charge the battery, thus, the LDO. But even then, you should be safe with the LDO and everything will work fine. It has no problem with 4.2 V at its output; that what the system would put there; at least not as long as it has 5 V at its input. 

Finally, if your system expects a smart battery (i. e. a battery that can communicate via a two-wire SMB bus), it depends on its software whether it will startup if no communication is established. But in any case, you can try.