What kills my LED lights?
ZnCl R6 batteries have a slightly higher ESR resistance than Alkaline such that variations in LED Vf from 2.9 to 3.1 will see additional ESR to limit the current. each White LED is about 15 Ohms @ 20 mA and rises with lower current and lowers with better quality LEDs, higher power LEDs and also the threshold of Vf lowers about -2 to 3 mV / ‘C rise in junction temperature.
As the power supply is regulated it will have a much lower output impedance than the LEDs or battery @ 3V. (milliohms)
You are witnessing  a slow thermal runaway response with the LED current and power dissipation rising every degree Celsius.
New concept
This is something like the Barhausen Criterion for oscillation, except here for DC. For each ‘C rise in junction temp,the threshold voltage part of Vf drops say 2.4mV but due to fixed voltage of regulator and Rbulk = 15 ohms of say 1 LED, the current increase. So with a 100’C rise in Tj, Vf has dropped 240mV and the added current is 240mV/15 ohms= 16mA above say 20mA, thus exceed max rating of 30mA.  Now the power has risen by 3V * 16 mA = 48 mW resulting in another more temperature rise.  We know a small THT transistor is about 200’C/W so a thicker epoxy case for a T5 LED Tja~ 300’C/W (FWIW) . Thus Tj continues to rise slowly another 48mW * 0.3’C/mW= 14’C
This then causes the ESR to reduce as current rises from the drop in Vf again. So thermal runaway starts slow and then finishes exponentially fast.
My Barkhausen Criteria variation here is the critical point for thermal runaway is where the source impedance is elevated to make the thermo-electrical loop gain=1.
It can be fixed by reducing the voltage to 2.9V anyway possible.
If there were only 1 LED then 100 mV/20 mA approx is 5 Ohms. If 2 LEDs in  Parallel it gets dimmer and drops to 2.85V approx. but now avoids the thermal runaway as the rise in Tjcn and mV drop in Vth of the LED (asymptote around 2.8V) is offset in current limit by the added 5 Ohms in series which is about the ESR of a ZnCl pair of R6 batteries.
Short Answer

add 5 Ohm approx. series 1/8th W resistor from 3V supply.

Assumptions
R6 batteries ESR are about 2 to 3 Ohms each
-(+/-50%)

LEDs are white 2.9 to 3.1V @ 20mA ( modern HB type) not 10 yr old styles.

voltage regulator is 3.0V and stable