Li-Po/Li-Ion solar charger and 3.3V stabilized output
I suggest you to first consider:


Estimate of how much power your application may draw from the battery.
Space constraints, if you have some, in order to better select the solar panel.
A DC-DC converter is the best choice when converting energy, because the switched components can achieve a very high efficiency. The most applications in solar are build with a buck converter, because the solar panels has usually more voltage than batteries voltage.  A buck-boost converter instead, is a device that can adapt a both higher or lower input voltage to the output fixed value.


I suggest you to consider open-hardware projects first to make a prototype and select the components (battery, solar panel).

Then you can look for the free schematic files and study the circuit for the design of the embedded PCB. Remember: even if you have a schematic you need skills to design a PCB layout, especially for switched circuits.

A good charger is SunnyBuddy from Sparkfun, sporting a Linear Instruments LT3652 switched buck charger with MPPT (passive) capabilities. It's a well documented project. But it needs a solar panel capable of at least \$7\text{ Voc}\$ to start properly.

If you want to have your life easier you can select an easy-to-find \$6\text{ V}\$ solar panel and use it with a linear charger with good efficiency, specifically hacked for solar. Here you can find the project based of Texas MCP73871.

Now you have a charger. You will need also a device to lower the battery's voltage to feed the ESP8266. You may consider an LDO component to simply lower the voltage from \$3.3\text{ V}-4.2\text{ V}\$ to \$3.3\text{ V}\$. This is the most simple choice but not the more efficient.

Or you can opt for another DC-DC buck converter. This Adafruit project has a nifty \$3.3\text{ V}\$ output with a 90-95% efficiency.

You can ask Adafruit support to provide you the schematic as it is not published yet.