Low power ESP8266 LDO or switching regulator
Good discussion already. I'll summarize the linear option as an answer.


A datasheet for the ESP8266 says that it can tolerate from \$3.0\:\textrm{V}\$ to \$3.6\:\textrm{V}\$.
A 1S Li-Po starts out almost \$4.2\:\textrm{V}\$ with low load and is nearly empty by about \$3.4\:\textrm{V}\$.


That leaves only about \$100\:\textrm{mV}\$ of headroom towards the end of battery life and that's probably not enough. It's almost certain that you'll need at least twice that much, using an LDO at \$70\:\textrm{mA}\$ and perhaps three times it at the initial \$350\:\textrm{mA}\$ you mentioned.

But it still may be possible to consider using a linear regulator, if you choose to operate at a lower voltage. One that is at or below \$3.1\:\textrm{V}\$. Sure. That's not much wiggle room. But at least it's non-zero. Still, you may also have to consider what else surrounds the ESP8266, too. And you've asserted the need for \$3.3\:\textrm{V}\$ but you haven't disclosed whether or not perhaps \$3.0\:\textrm{V}\$ or \$3.1\:\textrm{V}\$ could be successful. (Okay. I don't know of a fixed \$3.1\:\textrm{V}\$ linear LDO with low quiescent current. So probably \$3.0\:\textrm{V}\$.)

Given all of the above, all I can suggest regarding a linear LDO option is something like the LT1763. It may meet your needs. It's quiescent current is about \$30\:\mu\textrm{A}\$, too, and requires at most \$300\:\textrm{mV}\$ of overhead for your case. So perhaps.



Given a short search (you really should do your own), I did find this from TI: TPS783xx. The spec is below your peak startup current, but well above your operating current. And the quiescent current is extremely low, I believe.