Power source project
You do need to provide those devices with the correct voltages. There are a couple of solutions. 

You can hook the cells up in a series of 2 or 3, and then use buck regulators, but that may complicate your charging since you now need a balance charger.

Or you can hook them up in parallel, and then use a boost converter for the 5v, and some kind of LDO regulator for the 3.3v. I would go with this, because it allows you to throw in a generic 1s lipo charger without complications.

Examples of regulator boards you might use on a prototype:

Boost Regulators: https://smile.amazon.com/Crazepony-Voltage-Module-Inductris-Battery/dp/B01LW57OUM/

Charger+boost regulators: https://smile.amazon.com/Li-ion-Lithium-Battery-Charging-Module/dp/B07JNT728T/

3.3v Buck/Boost Regulator: https://smile.amazon.com/Pololu-Step-Up-Step-Down-Voltage-Regulator/dp/B01M5CRLOF/ (I will warn that these burn out very easily).

If you have trouble finding a regulator that can supply 3.3v from 3.7, you can always pull 5v from the boost reg and then step that down to 3.3. Not optimal, but should work in a pinch.