What does the RPC call "getmemoryinfo" show?
It is reporting the memory statistics of the locked memory pool, not all memory used in general.
The locked pool is used to store private keys and other security-sensitive material. Bitcoin Core requests from the OS that the memory used by this locked pool is not swapped out, to prevent it from ending up on a swapfile/swap partition on disk.