Number of possible functions for a PRP
Yes, that's right. This is also why the key size can be much larger than the block size. The idea of a (well-distributed) key is that it pseudo-randomly chooses a function from X to Y.

So for instance AES can have a block size of 128 and a key size of 256 (and it has) because there are "only" $2^{256}$ keys while there are $2^{128}!$ possible functions / permutations. This is more explicitly explained here.