Do lithium batteries 'explode'/'burn' because of the electric energy stored within?
"Electrochemical" shows the correspondence between chemical and electrical energy. The energy in a cell can be released slowly over time, or rapidly when short-circuited internally or externally, or the ingredients meet under the wrong conditions. No additional chemicals, such as oxygen from the air, are needed.

You state, "there wouldn't be enough electrochemical energy in the battery to cause such an intense reaction." Calculate how much energy is stored, instead of guessing:

The common Li-ion 18650 cell stores about 2.5 ampere-hours at 4 volts (rounded). That's 10 watt-hours, and multiply by 3,600 sec/hr to get 36,000 watt-seconds or joules (J). That is a lot of energy compared to some projectile weapons. For example, the largest cartridge shown in this reference, for the .50 caliber Browning Machine Gun, packs 18,000 J, half that of the inexpensive 18650 cell.

So the question is, does the energy release come from thermal effects due to ohmic heating, or from the direct combination of chemicals. Most likely, once the cell is physically damaged, whether by overheating or mechanical stress, direct chemical combination takes place. Though it can be exacerbated by water or atmospheric oxygen, there is more than enough energy in even small Li-ion cells to cause spectacular fires or explosions.