What are the connection between bandwidth, Nyquist criteria and quantization?
Shannon-Hartley Theorom proved the maximum Channel capacity,C and  Channel Bandwidth, B.

C/B=Log_2(1+S/N)=C/B

for BER of 1e-6 results in Ref indicate ;   

BPSK needs 11 dB SNR @ C/B=1
64-QAM needs 27 dB SNR @ C/B=6.33
1024-QAM needs 39 dB SNR @ C/B=8 excluding 20% overhead for FEC.   

Ref

https://www.csie.ntu.edu.tw/~hsinmu/courses/_media/wn_11fall/rate_adaptation.pdf
http://www.scte.org/TechnicalColumns/12-10-01%20spectral%20effieciency.pdf
https://www.embedded.com/print/4017668
http://www2.ensc.sfu.ca/people/faculty/cavers/ENSC805/classnotes/c4p3.pdf