Low recognition rates using GMM for image classification
Are you trying to model the distribution of the SIFT descriptors using GMM? Are you using the actual 128-element descriptors?  If so, then you are trying to model a distribution in a 128-dimensional space, which is not likely to work well.

You need to reduce the dimensionality. If you insist on using GMM, then the simplest thing to try is to use principal components analysis (PCA) to reduce the dimensionality of your descriptors.  I would not use more than 10 dimensions.  Here is a paper, that tried kernel density estimation on reduced SIFT descriptors among other things.  

Alternatively, you could try other algorithms, such as Bag Of Features, or the Pyramid Match Kernel.