Default-on, latch-off, MOSFET power switch
simulate this circuit &ndash; Schematic created using CircuitLab

Q1 and Q2 together are actually an SCR, but CircuitLab doesn't have an SCR symbol. This is essentially a crowbar circuit, with the input left unconnected.

F1 should be some sort of current limiting device. You said "stay off" but didn't specify for how long. A fuse will stay off for a long time. Replace this with a resistor and it will stay off until V1 decreases enough for the SCR to unlatch.