Are too low/changing voltages damaging (to Arcade PCBs)?
Your +5v doesn't need to be spot on, but it should be within a reasonable tolerance.  If your chips are spec'ed at 4.5 to 5.5v (which is common for logic series like the 74HCTxx family) then I would try to keep the 5v supply within ±5%, or 4.75 to 5.25v so you are comfortably within the chip limits.  Using this criteria, your voltage of 4.85v would be acceptable.

In general, I think it is a little better to be under-voltage than over-voltage, as the latter may put more stress on parts.

As far as adjusting the voltage while the circuit is turned on or not, I am going to suggest a slightly different approach.  Assuming you are making the adjustment using a potentiometer of some sort, using a screwdriver, if you turn the power supply off to make the adjustment, turn the pot a ways, and then turn the power supply back on you won't know for sure the voltage will still be within safe limits when it is back on.  So that is a bad idea.

If you can, it would be better to disconnect the PCB from the power supply.  Then you can adjust the voltage while leaving the power supply on, watching the voltage change with a multimeter.  Once you have the voltage adjusted the way you want, hook the PCB back up to the power supply (with it off of course), turn it back on and measure the voltage again. Your power supply should be well-regulated, but it is still possible for it to droop a bit under the full load of the system.  In that case, you can make teeny adjustments the supply to bring it into spec.