Would having no power supply but a large input signal damage the amplifier?
In general it is bad practice to apply input signals to a device before applying power. When you do this, you can cause semiconductors to 'latch up'. The semiconductor can act due to its physical build as a SCR (Silicon Controlled Rectifier), which can cause it to short out its pins. Subsequential application of power can cause excessive currents which destroy the device.

This does not only apply for single chips, but also for modules. Recently chips have been designed with internal protection diodes, but I doubt you want to rely on them for this. Modules that are hot swapable can most of the time be recognized by the fact that the power supply pins are slightly longer than signal pins, enforcing power to be applied before any signal.