Should I power my device using USB 5V or PC ATX PSU 5V? Crazy idea: use both?
Certainly NOT connect the PSU's 5 V to the USB 5V. The whole idea of the USB connector is providing power to peripherals which potentially misbehave. So the USB port should remain separate.

USB normally provides 100mA (which should be plenty for normal microcontrollers). Intelligent USB peripherals can negotiate up to 500mA by software.

The PSU can probably deliver over 15-20A on the 5V line, and somewhat less on the 12V line.

Note that the microcontroller's firmware has to be behave differently if it's selfpowered or draws from the USB port. The firmware has to inform the host on how it is powered (over the bus. This is part of the descriptor sent to the host when inserted). Frequently, USB devices feed from the USB +5V just for the communications, and use proper power for the rest.

Also, the peripheral has to disconnect its bus connections automatically when the USB +5V disappears, so it has to monitor that line.

I'd suggest that, unless you're wanting to heat coffee or other high current app, you use the USB power. It's less likely to interfere with the normal working of the PC. (And there might be an unused connector on the motherboard)