Maintain a Switch Position until Power Off?
I believe you could use a  inverted gated SR latch.



But that would be a bit complicated.

If you want a simple circuit only, you would need to acquire some 'unusual' components.



The MOSFET acts like a relay where it enables and disables the connection to the thyristor. The thyristor acts like a normal switch but if you leave it floating, it keeps its previous state, that is unless you disconnect the power supply.