Can I shut off the water supply by unplugging the water softener?
The power to the softener is only used to run the backwash and recharge cycle. It will continue to operate without power, and will even continue to soften water for a while (though it does eventually need to be recharged, depending on the hardness of your water and how much you use).



Certain types of bypasses can be used as shut-off valves. 

A custom-built one with 3 valves is easy: just close all valves.



There's a type with two valves that can also be used: close one (so the valves are in opposite directions).






Other types that only have one valve/control handle don't shut off the water flowing through them. 







Separately, you should have a main-shutoff that is reliable. If you have a leak in your house it will likely cost much more than a valve will. Install a ball (quarter turn) valve, rather than a globe or gate valve -- very reliable, and will easily last at least a couple decades.