What are the types of noise (FM,AM, MSK modulated) that I can use for jamming GSM signals?
Almost any kind on in-band interference (like Gaussian noise) will mess up the GSM signal and prevent its correct reception. Note, however, that:


Transmitting on the GSM band without a license is illegal in many locations.
Tampering with cell phone operation in general is also illegal.


If you insist on doing this, one approach would be to set up a GSM base station using OpenBTS in an environment where your signals won't interfere with other base stations and won't be otherwise detected (for example in a laboratory with RF shielding). You can use unlocked phones to connect to your OpenBTS base station, and then measure the effect of your interefernce on their performance.