Lithium ion (Li-ion) vs. Lithium thionyl chloride (LTC) battery?
I had to use Lithium Thionyl battery due to higher temperature range. That's the only advantage I've found for this technology.
I would advice you to keep Lithium-ion technology as long as your environmental constraints are "standard". And by standard, I mean the kind of constraints a human could endure without suffering health problems.

If you want to be sure that your voltage stay "constant", the only info you need is the standard discharge rate of your battery. Very stable voltage mean LDO in most cases. Choose an LDO with a maximum rated current close to the maximum rated consumption of your system.
I would advise to use a good decoupling architecture based on vendor recommandations and maybe double the capacitor capacity from what they recommand.

If I would be very picky, I would study every different BMS (battery management systems) I could use for a single cell architecture. Some of them can influence your peak available current. And as a personal habbit, I dislike to put a battery directly in front of my functional composents.