What are some general tips to improve my MNIST classifier?
You may find this OpenML interesting
In there several benchmarks on different datasets are shown so that you can see how different models score across the same dataset
For your task MNIST the plot shows:

that might give a good idea of how well your model is performing as compared to other implementations/models
Metric button in the upper left corner will allow you to compare in terms of classification metrics and also in terms of execution time