Infinite loop on microcontroller vs modern CPU
Why is this seemingly ok on a microcontroller but not usually wanted on a microprocessor? 


It is also unwanted on a microcontroller for the same reason: it wastes power.


  Am I right in thinking that the ATmega is in fact running at 100%


Correct.


  and that because it is so low powered it doesn't cause any obvious heat problems?


Correct. However, if you run your microcontroller on batteries, then you have to think real hard about not wasting power. On a tiny cpu like AtMega328P it will not cause heat problems, but it will definitely shorten battery life.

All cpu's, whether they're desktop powerhouses or tiny microcontrollers, use the same methods:

1- Reduce clock speed or voltage.

2- Shut down unneeded hardware.

3- Go to sleep and wake up on an event (this is a special case of shutting down unneeded hardware, in this case the cpu is shut down).

In the AtMega328P you can implement this too. You can use a slower clock if you don't need all the awesome power of the 8-bit core, you can shut down unneeded peripherals... and the most important is the sleep mode.

You should read the manual for details, as there are several sleep modes which differ in wake-up latencies, what peripherals remain online and able to wake the cpu, whether RAM data is conserved or lost, etc. But basically the idea is: when in sleep mode, the cpu is stopped so it uses much less power. When an interrupt occurs, this wakes up the CPU and it processes the interrupt.

Of course you have to use the proper sleep mode and configure it properly so the peripheral that needs to wake up the cpu (for example, a timer or a GPIO interrupt) is not shut down. If everything is shut down, you'll have to use NMI or even Reset to wake it up, in the latter case by rebooting it.

If all your application does is wait on interrupts, like:


Pin Change Interrupt (PCI) to detect a button press or an incoming signal
Timer
Data received by UART or USB
etc


Then you don't need to spin the main loop. After configuring everything at boot, you'd start the main loop with a "go to sleep" instruction. The next instruction will execute after the cpu wakes up, processes all pending interrupts, and returns to the main loop. Then the main loop can, if required, do something about the received events, if they were not entirely handled by the interrupt code... and then go back to sleep.

Even if you're not using batteries, having low standby current can a mains powered the switching power supply to skip cycles and waste a lot less power too.