Directly Power STM32 from Lithium Battery
If your goal is the longest runtime, use a switching buck regulator to efficiently drop the battery voltage to 1.8V (or 2.4V if you require the full radio power output).

If your goal is decently long runtime without the complexity of a switch-mode supply, use an LDO regulator to drop the voltage to the same level.  The microcontroller and regulator will most likely draw less current at the lower operating voltage, compared to running them directly on the supply.

If you want minimal extra components and consider an LDO + input &amp; output capacitors to be too many, then you could place a 3.6V Zener diode in parallel with the supply.  That would ensure your ICs don't get exposed to out-of-spec voltages.  When the battery is really full and has an open-circuit voltage above 3.6V, the Zener will conduct and the "excess" voltage will be effectively dissipated in the battery's internal resistance.