GPIO state of MCU (AVR) with no supply voltage
No, the inputs are not floating, and yes, applying a voltage to them when power is off can lead to damage.  This is true for just about any modern IC, where the data sheet does not explicitly say otherwise.

In the case of say an ATmega328p, we can see it in the "Absolute Maximum Ratings" section


  Voltage on any Pin except RESET
  with respect to Ground .  -0.5V to VCC+0.5V


These limits are typically due to the presence of ESD protection diodes which divert currents on the I/O pins to the supply rails.  When the supply rail is at ground potentially, the allowed input range is just less than a diode drop higher or lower.   Applying a higher voltage (at least without large series resistance) would cause the protection diode to conduct, and could easily provide enough current to damage it.  Additionally, it would attempt to raise the entire board's supply net.  Even if this did not cause damage it can lead to putting various circuitry into a state from which power on reset would not cleanly work (for example, while it's not a problem the AVR seems to have, I've seen other systems which would not boot if a USB-TTL adapter's data lines were connected before the system was powered, even if that adapter was connected via substantial series resistance)

If you need a defined level, that level should be ground via a pulldown resistor.  Even better, also powerdown whatever the MCU is connected to.

Remember also that even if the pin is not floating when power is off, it will be floating between reset and when it is configured as an output.  Especially if something like a serial bootloader is involved, that could potentially be a substantial period of time.