title: Image classifier using cifar 100, train accuracy not increasing
body: <p>I was trying to train an image classifier model using cifar100 dataset in tensorflow, but accuracy is not increasing over 1.2%. I googled the issue and found several solutions but still my model is not doing well.</p>
  
  <p>I implemented a few steps such as:</p>
  
  <ol>
  <li>increasing CNN layer and pooling along with drop outs and 
  normalization</li>
  <li>changing no. of dense layers </li>
  <li>changing batch size and epochs</li>
  <li>changing optimizers</li>
  </ol>
  
  <p>A common thing I noticed is that with epoch=10 and batch size=256 &amp; epoch=500 and batch size=512 training loss and accuracy is varying in the same way.</p>
  
  <p>To prevent over-fitting I also tried dropout regularisation, this shows some change (train acc. varies in between 0.5 and 1.2%), with the same parameters when I increased epochs nothing changed(train and model acc.)..</p>
  
  <p>I wanted to know whether this is a problem with the dataset or with the model definition.</p>
  
  <p>classifier model: </p>
  
  <pre><code>def classifierModel(inp):
      layer1=tf.nn.relu(tf.nn.conv2d(inp, filter=tf.Variable(tf.truncated_normal([5,5,3,16])), 
                                     strides=[1,2,2,1], padding='SAME'))
      layer1=tf.nn.bias_add(layer1, tf.Variable(tf.truncated_normal([16])))
      layer1=tf.nn.relu(tf.nn.max_pool(layer1, ksize=[1,1,1,1], strides=[1,2,2,1], padding='SAME'))
  
      layer2=tf.nn.relu(tf.nn.conv2d(layer1, filter=tf.Variable(tf.truncated_normal([5,5,16,32])), 
                                     strides=[1,2,2,1], padding='SAME'))
      layer2=tf.nn.bias_add(layer2, tf.Variable(tf.truncated_normal([32])))
      layer2=tf.nn.relu(tf.nn.max_pool(layer2, ksize=[1,1,1,1], strides=[1,2,2,1], padding='SAME'))
  
      layer3=tf.nn.relu(tf.nn.conv2d(layer2, filter=tf.Variable(tf.truncated_normal([5,5,32, 64])), 
                                     strides=[1,2,2,1], padding='SAME'))
      layer3=tf.nn.bias_add(layer3, tf.Variable(tf.truncated_normal([64])))
  
      layer3=tf.nn.relu(tf.nn.max_pool(layer3, ksize=[1,1,1,1], strides=[1,2,2,1], padding='SAME'))
      layer3=tf.nn.dropout(layer3, keep_prob=0.7)
      print(layer3.shape)
  
  
      fclayer1=tf.reshape(layer3, [-1, weights['fc1'].get_shape().as_list()[0]])
      fclayer1=tf.add(tf.matmul(fclayer1, weights['fc1']), biases['fc1'])
      fclayer1= tf.nn.dropout(fclayer1, keep_prob=0.5)
      fclayer2=tf.add(tf.matmul(fclayer1, weights['fc2']), biases['fc2'])
      fclayer2=tf.nn.dropout(fclayer2, keep_prob=0.5)
      fclayer3=tf.add(tf.matmul(fclayer2, weights['fc3']), biases['fc3'])
      fclayer3=tf.nn.dropout(fclayer3, keep_prob=0.7)
      outLayer=tf.nn.softmax(tf.add(tf.matmul(fclayer3, weights['out']), biases['out']))
      return outLayer
  </code></pre>
  
  <p>Optimizers, cost, accuracy: </p>
  
  <pre><code>cost=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=model, labels=y))
  optimizer=tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
  correct_pred=tf.equal(tf.argmax(model, 1), tf.argmax(y, 1))
  accuracy=tf.reduce_mean(tf.cast(correct_pred, tf.float32))
  </code></pre>
  
  <p>Training:</p>
  
  <pre><code>with tf.Session() as sess:
  sess.run(init)
  for i in range(epochs):
      #shuffle(idx)
      #train_features=train_features[idx, :, :, :]
      #train_labels=train_labels[idx, ]
      for batch_features, batch_labels in get_batches(batch_size, train_features, train_labels):
          sess.run(optimizer, feed_dict={x:batch_features, y:batch_labels})
      if (i%display_step==0):
  
          epoch_stats(sess, i, batch_features, batch_labels)
  
  model_acc=sess.run(accuracy, feed_dict={x:test_features, y:test_labels})
  saver.save(sess, save_file)
  
  writer.add_graph(sess.graph)
  </code></pre>
  
  <p>Results:</p>
  
  <ol>
  <li>epoch : 0    - cost : 4.62     - acc: 0.01 </li>
  <li>epoch : 1    - cost : 4.62     - acc: 0.01 </li>
  <li>epoch : 2    - cost : 4.62     - acc: 0.008</li>
  <li>epoch : 3    - cost : 4.61     - acc: 0.012</li>
  <li>epoch : 4    - cost : 4.61     - acc: 0.005</li>
  <li>epoch : 5    - cost : 4.62     - acc: 0.006</li>
  <li>epoch : 6    - cost : 4.62     - acc: 0.016</li>
  <li>epoch : 7    - cost : 4.62     - acc: 0.012</li>
  <li>epoch : 8    - cost : 4.61     - acc: 0.014</li>
  <li>epoch : 9    - cost : 4.62     - acc: 0.009</li>
  <li>Model accuracy - 0.010499999858438969</li>
  </ol>