How to work on different models for a given problem? I am working on the MNIST data on my own. The idea is to use different values for the number of hidden layers, number of nodes in a given layer, etc. How do you organize these things while you are working on creating a model for a problem? DO you do everything in one code file or you use different code files for choosing the best?
it seems to me that you are talking about hyperparameter tuning and effect of hyperparameters on the network in general. If you are working with tensorflow, I recommend you to look into tensorboard.

Hands-on TensorBoard can be a good starting point.