Can ESD damage a board that is not powered up?
My question is, if the PCB does not have the power supply connected,
  can ESD still damage the PCB or any of the sensitive components?


ESD doesn't need a pathway back to ground to be destructive, almost everything looks like a conductor when voltages are in the thousands of volts. Once an ESD event gets started it causes a current avalanche and materials (even air and insulative materials) start to break down and become conductive and destroys anything in its current pathway. 


Source: https://nepp.nasa.gov/index.cfm/6095

The picture above shows a capacitor on an IC, which has a layer of insulation between it and the other side of the capacitor. The ESD event blew right through the insulator.