How to query all utxos of an address from a full node?
There isn't really a way to do this efficiently, as Bitcoin Core is not designed for that.

You can search using the the scantxoutset RPC command, but all that is doing is iterating over the entire database and returning matching entries (it's more powerful than just by address though; it can for example search for an xpub + derivation path too).

There is some work around adding support for an optional (and separate) address-based index to Bitcoin Core, but that will at the earliest be for v0.21.

Other than that, there are a number of other software packages that can build such an index (independently from Bitcoin Core).