How does a router send different data to different devices simultaneously?
Does the router do something like send addressed data packets and each device filters out the packets meant for them?

Yes - this is what the MAC address is for.
(There is some complexity with encryption, but in practice it's possible to decrypt packets sent to other devices.)

Or is the router able to send different info/EM waves to different devices simultaneously?

Yes - this is called MIMO.
MIMO is an extremely complicated system, but it relies on the way interference and reflected signals interact. All the signals go everywhere, but at the target device the signal is strongest.