What is the truth about 1.5 V "lithium" cells?
Lithium batteries come in many different chemistries, and it is the chemistry that governs the voltage. The most common chemistries are on the order of 3-4V, but there are chemistries which have a 1.5V terminal voltage.

The wiki page for Lithium batteries has a list of many different chemistries and their voltages. A Lithium anode with an Iron Disulphide cathode (\$\mathrm{Li-FeS_2}\$) is one such example of a 1.5V terminal voltage, and is the chemistry used in the AA replacement batteries as per the datasheet link on the Wiki page, and in @pjc50's answer.