Annoying "unmount" notification when removing SD card
Unmounting an SD card is comparable to the "Safely remove device" feature of Windows, which should be used when removing USB drives etc. 

If you remove the device without notifying the OS that the connection is going to be lost, some data may not be written to the device, and corruption and data loss can occur. This is because the OS uses buffers to store the data before transferring it to the device itself. 

Even though the operation appears to be complete, there still may be some data left in the buffer and waiting to be written to the device. Unmounting/safely removing the device forces there buffers to be written to the device before removal, thus preventing corruption.

I'd recommend either getting used to going through the menus to unmount the SD, or if your device supports, using a widget like Eject SD for easier access. Powering off the device works too, but as you mentioned, it's a bit slower.