In a cryptosystem, why does the message space need to be finite?
One reason for this is that one can first define a cryptosystem having a small finite message space, and then on top of that, construct another cryptosystem that can handle longer message spaces. This is a very reasonable modular design methodology.

A couple of examples:


given a block cipher with a finite message space, one can use a 
block cipher mode (like CBC or CTR) to extend the message space
to be essentially infinite.
given the Paillier public-key cryptosystem, one can institute a
specific length-parameter (as done by Damgård and Jurik) and allow
to efficiency encrypt messages of arbitrary length.


One caveat though is that if messages are of arbitrary length, then also the time to encrypt increases accordingly. If we want to be the cryptosystem to be secure against adversaries working in some time $t$, it is better to restrict the message length so that encrypting any messages of that length takes considerably less time than breaking the cryptosystem (with a small message length).

Remark: one can obviously try to define security in a way that breaking small messages can be easier than breaking long messages, but this does not make sense in practice.