How can I verify certificate signature algorithm in TLS 1.3 using PSS? I've read this post but I'm still unable to understand some things.

I'm trying to verify that the signature algorithm used in the certificate matches the proposed list from the client. (As it's mandatory in TLS 1.3)

But I cannot understand the difference in verification when the client offers both signature schemes 'rsa_pss_rsae_' and 'rsa_pss_pss_'.

Both of them indicate that the signing can be in PSS, but from my understanding, the 'rsa_pss_rsae_' is specifying constraints on the public key? Is that correct?

Does it mean that when verifying the certificate signature algorithm, both will be signed with the OID RSASSA-PSS BUT the difference will be in the public key?

(as in RSA (rsaEncryption) or PSS only (RSASSA-PSS) key)
It depends on what publickey is. An X.509/PKIX cert contains a SubjectPublicKeyInfo structure which contains an AlgorithmIdentifier that specifies what type the key is and in some cases to some extent how it is used, plus the actual publickey value (wrapped in an OCTET STRING type). 
See also the links to 3279 and 4055 (which is updated by 5756).
These TLS1.3 SignatureScheme codes (both the RSA_PSS_RSAE set and the RSA_PSS_PSS set) specify that in the cert that contains the publickey usable to verify the PSS signature (often imprecisely called the signing cert) the SPKI AlgorithmIdentifier use a particular OID; they do not constrain the actual publickey value. Is that a constraint on the publickey? You decide. 

For the signature_algorithms_cert extension, yes for PSS-signed certificates the two AlgorithmIdentifier's for the signature algorithm in the child cert will use the OID id-RSASSA-PSS and applicable parameters, while the parent cert's SPKI is constrained as above. When the signature_algorithms extension applies to protocol signatures rather than cert signatures, the protocol signatures are not identified by any OID (see 4.4.4) and the leaf cert's SPKI is constrained as above; this is equally mandatory.