Voltage limiter is shutting down completely when tripped
As drawn, your circuit should be able to hold it's output voltage for several seconds.

The potentiometer P1 does provide a leakage path though, with a time constant of about 50 s. This means you'll see the voltage droop noticeably in just a second or two. It should take about 2 min to get down to 1.5 V, though.

Once it drops far enough, Q1 should be disabled, causing the voltage to rise again. This cycle would continue, resulting in an oscillating output voltage. If you're measuring the output with a multimeter, it's possible this is happening (but much faster than expected) and you're just seeing the average value of the oscillating voltage, rather than the value from any particular instant in time.

If you have any load attached to the terminals on the right, that will also speed up discharging the capacitor.

Another possibility, if you wired the capacitor in reverse, there would be substantial leakage current through the capacitor itself. I'm saying this because at 1 mF, you're almost certainly using either an aluminum or tantalum electrolytic (or a bunch of them in parallel, in which case it would only take one wired wrong to mess things up).

Edit: Also look at the leakage current spec on your capacitor. Looking at a few different 1 mF parts, it's quite easy to find one with leakage current equal to or larger than the drain you're allowing through P1.