MNIST - Vanilla Neural Network - Why Cost Function is Increasing?
Tensorflow's softmax function only works if the number of batches are in the rows and the output in the columns. If these are reversed, then you need to transpose the tensors in the cost function.