Relay(or Alternative of relay) without pull-back feature
What the question describes is called a latching relay:



There are several types of latching relay manufactured: NO/NC/NO+NC, SPDT/DPDT, timed reset, switch reset, and so on. 

Depending on required rating and load type, latching relays are available in tiny sugar-cube type PCB mounted packages, all the way up to industrial versions. 

Also, they come in both solid state, and electromagnetic types.