How do these modern power buttons on devices work
The modern switches don't contain any magic.  In fact, they are less complicated and expensive than real physical on/off switches.

These switches are just inputs to a microcontroller.  The microcontroller can tell when you push the button, and the rest is policy encoded in the firmware to decide what to do about it.  The power is usually switched with transistors.  This means the button itself doesn't have to handle high voltage or high current, so there are a lot more options to make it and for it to be small.  It could be a membrane switch, for example, which you'd never use to switch wall power.

This does mean that a little bit of the device is usually on, at least enough to power the microcontroller.  However, modern microcontrollers can take such tiny amounts of power when doing nothing but waiting for a switch signal that this power is irrelevant in most cases.

In some cases, the button actually causes the micro to get powered up when pressed, which then turns on some transistors or a relay or something to keep the power on.  When you press the button to turn the device off, the micro shuts down everything, including itself.