Why some computer peripherals should not be disconnected without turning off things?
I think it is threefold:


When power supply of various IC's is not connected before signals on other pins, the semiconductor can act as a thyristor between power supply rails, which results in short circuit the PSU (failing the whole PC) and high power dissipation or blowing the chip(s on the device).
The computer for many devices just isn't aware of them being plugged or unplugged. The operating system will not be able to use it as it is unaware of its existence OR the operating system is trying to use a device that is no longer available. Only few buses are able to cope with hot plugging (eg. USB).
(inductive) power surges due to unplugging can easily blow poorly designed hardware. I blew out the DAC's on my video processor once...