I have implemented a CNN with images as input and 101 classes as output.I have applied mean subtraction and normalization to the input before giving it as input to the network.I have also experimented with multiple optimizers (ADAM & SGD) and also changed the learning rate (0.1,0.01,0,001) and also batch sizes (19,32,64,128,200). But with all of these cases I had observed a similar pattern. I dont think the network is learning much and more importantly the accuracy oscillating between a few values. I had initially thought it was because of the batch size but I was wrong.I have built and trained the exact same network with same inputs with caffe.  And achieved a 55% accuracy(check for working of  network model). My guess is the implementation went wrong somewhere but not sure where.MycodeOutputThis particular output is with lr=0.001 batch_size=200 and ADAM optimizerIter 2000, Minibatch Loss= 12560.197266, Training Accuracy= 0.01000Testing Accuracy: 0.025Iter 4000, Minibatch Loss= 4587.021484, Training Accuracy= 0.00500Testing Accuracy: 0.005Iter 6000, Minibatch Loss= 2196.316406, Training Accuracy= 0.02000Testing Accuracy: 0.02Iter 8000, Minibatch Loss= 918.784119, Training Accuracy= 0.01500Testing Accuracy: 0.0Iter 10000, Minibatch Loss= 315.327911, Training Accuracy= 0.01000Testing Accuracy: 0.0Iter 12000, Minibatch Loss= 102.744026, Training Accuracy= 0.01000Testing Accuracy: 0.01Iter 14000, Minibatch Loss= 59.417763, Training Accuracy= 0.00500Testing Accuracy: 0.005Iter 16000, Minibatch Loss= 33.026432, Training Accuracy= 0.00500Testing Accuracy: 0.02Iter 18000, Minibatch Loss= 30.361868, Training Accuracy= 0.02000Testing Accuracy: 0.005Iter 20000, Minibatch Loss= 33.944790, Training Accuracy= 0.01000Testing Accuracy: 0.02Iter 22000, Minibatch Loss= 15.029477, Training Accuracy= 0.02000Testing Accuracy: 0.02Iter 24000, Minibatch Loss= 13.751340, Training Accuracy= 0.02500Testing Accuracy: 0.02Iter 26000, Minibatch Loss= 9.085027, Training Accuracy= 0.01500Testing Accuracy: 0.015Iter 28000, Minibatch Loss= 14.407433, Training Accuracy= 0.02000Testing Accuracy: 0.02Iter 30000, Minibatch Loss= 11.090684, Training Accuracy= 0.02500Testing Accuracy: 0.02Iter 32000, Minibatch Loss= 9.385777, Training Accuracy= 0.02000Testing Accuracy: 0.03Iter 34000, Minibatch Loss= 12.902111, Training Accuracy= 0.01500Testing Accuracy: 0.02Iter 36000, Minibatch Loss= 5.580408, Training Accuracy= 0.02000Testing Accuracy: 0.005Iter 38000, Minibatch Loss= 11.641799, Training Accuracy= 0.02000Testing Accuracy: 0.03Iter 40000, Minibatch Loss= 6.941562, Training Accuracy= 0.01500Testing Accuracy: 0.02Iter 42000, Minibatch Loss= 6.781301, Training Accuracy= 0.01000Testing Accuracy: 0.0Iter 44000, Minibatch Loss= 6.840708, Training Accuracy= 0.01000Testing Accuracy: 0.01Iter 46000, Minibatch Loss= 5.501908, Training Accuracy= 0.01000Testing Accuracy: 0.03Iter 48000, Minibatch Loss= 4.667253, Training Accuracy= 0.03000Testing Accuracy: 0.015Iter 50000, Minibatch Loss= 4.613351, Training Accuracy= 0.01000Testing Accuracy: 0.01Iter 52000, Minibatch Loss= 7.234328, Training Accuracy= 0.02000Testing Accuracy: 0.01Iter 54000, Minibatch Loss= 8.120478, Training Accuracy= 0.02500Testing Accuracy: 0.01Iter 56000, Minibatch Loss= 6.804164, Training Accuracy= 0.01000Testing Accuracy: 0.01Iter 58000, Minibatch Loss= 5.060484, Training Accuracy= 0.03000Testing Accuracy: 0.02Iter 60000, Minibatch Loss= 4.937673, Training Accuracy= 0.02000Testing Accuracy: 0.015Iter 62000, Minibatch Loss= 8.758166, Training Accuracy= 0.01000Testing Accuracy: 0.02Iter 64000, Minibatch Loss= 4.616434, Training Accuracy= 0.01500Testing Accuracy: 0.025
Tensorflow oscillating Test and Train Accuracy?
machine learning;python;neural network;convnet;tensorflow