Should continuity test be performed when the device is off? Why?
Such a test is completely meaningless, and potentially damaging to the meter.
If you want to test continuity or resistance, remove all sources of power, and discharge any stored energy sources.

FakeMustache has explained that, but simply put, the meter is SOURCING a (usually low) test voltage.  If you connect it to something that is already powered, you are connecting the two sources together, and the meter isn't designed to handle external sources in continuity or resistance (or capacitance, or inductance, or any other passive) mode.  

When a meter is set to voltage, it presents a very high impedance to the source being measured.  When set to current, it presents a very low impedance, but both are designed specifically to handle reading an active source.  When it is set to measure a passive circuit or device, it must source its own test voltage, and the precision regulated source is easily damaged by external power.