Do I need to power off the Arduino board before doing something?
It is not safe, as you may accidentally connect some points, which should not be connected and destroy Arduino and/or any other part of circuit. typically legs of resistors/LEDs tends to touch other legs near them, when you are putting it in - and if it is powered, magical smoke can happened easily.

Even if you do not this kind of mistake, not everything can be put in random order without risc of damage.

So everytime you change something with circuit, disconnect power source/USB first. And if you have big capacitors anywhere, you better make sure, they are discharged too.

You switch on the board, when the building is done and you want to program it, or use the already present program.