title: Tensorflow Layers API CNN parameters wont change during training
body: <p>I am new to Tensorflow so the problem might be really silly.</p>
  
  <p>I have been trying to write a simple CNN for a MNIST handwritten digits dataset using Tensorflow. The problem is that parameters are not updated by the Optimizer(being monitored by Tensorboard summary).<br>
  Graph seems OK, even though scopes created by Layers API seem weird. Gradients are computed from every layer.
  Please, help!</p>
  
  <p>I am using training data from here: <a href="http://yann.lecun.com/exdb/mnist/" rel="nofollow noreferrer">http://yann.lecun.com/exdb/mnist/</a></p>
  
  <p>Here is the code</p>
  
  <pre><code>import tensorflow as tf
  
  DATA = 'train-images.idx3-ubyte'
  LABELS = 'train-labels.idx1-ubyte'
  NUM_EPOCHS = 2
  BATCH_SIZE = 15
  #Data definition
  data_queue = tf.train.string_input_producer([DATA,])
  label_queue = tf.train.string_input_producer([LABELS,])
  
  reader_data = tf.FixedLengthRecordReader(record_bytes=28*28, header_bytes = 16)
  reader_labels = tf.FixedLengthRecordReader(record_bytes=1, header_bytes = 8)
  
  (_,data_rec) = reader_data.read(data_queue)
  (_,label_rec) = reader_labels.read(label_queue)
  
  image = tf.decode_raw(data_rec, tf.uint8)
  image = tf.reshape(image, [28, 28, 1])
  label = tf.decode_raw(label_rec, tf.uint8)
  label = tf.reshape(label, [1])
  
  image_batch, label_batch = tf.train.shuffle_batch([image, label],
                                                   batch_size=BATCH_SIZE,
                                                   capacity=100,
                                                   min_after_dequeue = 30)
  #Layers definition
  conv = tf.layers.conv2d(
    inputs=tf.cast(image_batch, tf.float32),
    filters=15,
    kernel_size=[5,5],
    padding='same',
    activation=tf.nn.relu)
  
  conv1 = tf.layers.conv2d(
    inputs=conv,
    filters=15,
    kernel_size=[3,3],
    padding='same',
    activation=tf.nn.relu)
  
  pool_flat = tf.reshape(conv1, [BATCH_SIZE, -1])
  
  dense1 = tf.layers.dense(inputs=pool_flat, units=30, activation=tf.nn.relu)
  
  output = tf.nn.softmax(tf.layers.dense(inputs=dense1, units=10))
  
  #train operation definition
  onehot_labels = tf.one_hot(indices=tf.cast(tf.reshape(label_batch,[-1]), tf.int32), depth=10)
  
  loss = tf.losses.softmax_cross_entropy(onehot_labels=onehot_labels,
                                         logits=output)
  
  global_step = tf.Variable(0,name='global_step',trainable=False)
  train_op = tf.train.GradientDescentOptimizer(0.05).minimize(loss, global_step = global_step)
  
  #Summaries definition
  
  for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='conv2d'):
      tf.summary.histogram(var.name, var)
  for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='conv2d_1'):
      tf.summary.histogram(var.name, var)
  for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='dense'):
      tf.summary.histogram(var.name, var)
  for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='dense_1'):
      tf.summary.histogram(var.name, var)
  tf.summary.image("inp", image_batch, max_outputs =1)
  loss_summary = tf.summary.scalar("loss", loss)
  summaries = tf.summary.merge_all()
  
  #init
  sess = tf.Session()
  summary_writer = tf.summary.FileWriter('log_simple_stats', sess.graph)
  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(coord=coord, sess=sess)
  sess.run(tf.global_variables_initializer())
  
  #loop
  for i in range((60000*NUM_EPOCHS)//BATCH_SIZE):
      sess.run(train_op)
      if(i%100):
          merged = sess.run(summaries)
          summary_writer.add_summary(merged, i)
  
  
  coord.request_stop()
  coord.join(threads)
  </code></pre>
  
  <p>EDIT
  Custom layers give same result.</p>
  
  <p>Custom layers definitions:</p>
  
  <pre><code>def convol(input, inp, outp, name="conv"):
      with tf.name_scope(name):
          w = tf.Variable(tf.truncated_normal([5, 5, inp, outp], stddev=0.1),name="W")
          b = tf.Variable(tf.constant(0.1, shape=[outp]), name="B")
          filtered = tf.nn.conv2d(input, w, strides=[1,1,1,1], padding="SAME", name="conv2d")
          activation = tf.nn.relu(features=(filtered+b), name="activation")
          tf.summary.histogram(name=w.name, values=w)
          tf.summary.histogram(name=b.name, values=b)
          tf.summary.histogram(name=activation.name, values=activation)
          return activation
  
  def dense(input, inp, outp, name="dense"):
      with tf.name_scope(name):
          w = tf.Variable(tf.truncated_normal([inp, outp], stddev=0.1), name="W")
          b = tf.Variable(tf.constant(0.1, shape=[outp]), name="B")
          act = tf.matmul(input, w) + b
          tf.summary.histogram(name=w.name, values=w)
          tf.summary.histogram(name=b.name, values=b)
          tf.summary.histogram(name="activation", values=act)
          return act
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>So after some time messing with this and MNIST examples from tf I have noticed that weights  are not being learned. The way I have handled the data reading has messed up something about gradient computation. I have just taped the class that reads MNIST dataset into my code and it works 100% without any tweaks to the parameters.</p>