Circuit that retains a state even after power loss
Yes. One thing I have done is to use a very low current fuse as a one-time programmable bit. You may need to parallel some drivers or use a transistor to get enough current.
For example, connect one end of the fuse to ground and the other end to a pullup resistor. As long as the fuse is intact the voltage across it is (nearly) zero. Blow the fuse, and the voltage across it will be the power supply voltage forever more.