Effect of NOT changing filter weights of CNN during backprop
By not changing the weights of the convolutional layers of a CNN, you are essentially feeding your classifier (the fully connected layer) random features (i.e. not the optimal features for the classification task at hand).

MNIST is an easy enough image classification task that you can pretty much feed the input pixels to a classifier without any feature extraction and it will still score in the high 90s. Besides that, perhaps the pooling layers help a bit...

Try training an MLP (without the conv/pool layers) on the input image and see how it ranks. Here is an example where an MLP (1 hidden &amp; 1 output layer) reached 98+% without any preprocessing/feature extraction.



Edit:

I'd also like to point out to another answer I wrote, which goes into more detail on why MNIST is so easy as an image classification task.