how to define an issue with neural networks
A quick advice may be to solve an intermediate task (e.g. to use your own 5x5 ASCII "pictures" of digits), to have more neurons in the hidden layer, to reduce the data set for quicker simulation, to compare your implementation to other custom implementations in your programming language.
I have built a system where the neural network can change size (number of and size of hidden layers, etc). When training it with a learning rate of 0.5, 1 hidden layer of 4 neurons, 2 inputs and 1 output, it successfully learns the XOR and AND problem (binary inputs, etc). Works really well. 
When I then make the structure 784 inputs, 1 hidden layer of 30 neurons, and 10 outputs, and apply the MNIST digit set, where each input is a pixel value, I simply cannot get good results (no better than random!). My question is quite theory based: If my code does seem to work with the other problems, should I assume i need to keep experimenting different learning rates, hidden layers, etc for this one? Or should decide theres a more underlying problem? 
How do I find the right combo of layers, learning rate, etc? How would you go about this?
Testing is also difficult as it takes about 2 hours to get to a point where it should have learnt... (on a mac)
No, I am not using TensorFlow, or other libraries, because I am challenging myself. Either way, it does work ...to a point!
Many thanks. And apologies for the slightly abstract question - but I know its a problem many beginners have - so I hope it helps others too.