Should I turn off my electric water heater when the water is off?
Yes, you should turn off power to the water heater, and make sure that all air is out of it before turning it back on.
Depending on its location in your house relative to various outlets, air getting into the tank may be unlikely, or very likely. But if sufficient air gets in the tank (typically from opening faucets to get a little bit of water that will dribble out, or because you forget the water is off and turn the faucet on) then the elements will burn out rather quickly from operating in air, rather than as designed in water.
Likewise, depending how long your water is off for, you may find use for the 30-50 gallons stored in a typical water heater, but in that case you surely need to make sure that air is well vented before turning it back on. That's basically opening hot taps and waiting until you get a solid stream of water, not bubbles or air, once the water supply is turned back on.