My Neural Network is not learning anything I try to train convolutional neural net for a classification problem. However, my neural net is not learning anything. It guesses only two labels and ignores the rest. Even though I try to train to overfit my neural net, the loss function is not decreasing at all. My I try to make my network go as deep as 12 layers of the convolutional neural net in order to overfit the subsampling data. However, it is not working. Do you have any suggestion what to look?
Make sure that while training, you are passing the data in a random order. It looks like you are passing all images of one class at a time then the next class etc. Shuffle the images and pass it in a random manner.