Aliasing in digital communication system
If the signal breaks the Nyquist criterion we'll only get a part of the signal.


This needs to be clarified: the frequencies above Nyquist will fold back into the Nyquist band, so it's not just that you lose part of the signal; it's that the signal is corrupted (usually) beyond hope of reconstruction.

As an example, consider a 4 kHz real signal sampled at 4 kHz -- the frequencies between 2 and 4 kHz will fold back into the 0 to 2 kHz range; not a single "part" of the signal will survive.

You're correct that this can't be solved by modifying the constellation order $M$ or the carrier frequency. As you correctly assume, you have two options. You can reduce the bandwidth through pulse shaping; if the symbol rate is $R$, then the minimum bandwidth $B=R/2$ is achieved with sinc pulses. The second option is to reduce the rate.

(As a side note, recently there have been proposals to increase the rate without increasing the bandwidth (search for "Faster than Nyquist signaling"), but I think these ideas are outside the scope of your studies at the moment.)