title: How to correctly use the tf.layers.batch_normalization() in tensorflow?
body: <p>
  I'm confused by the <code>tf.layers.batch_normalization</code> in tensorflow. </p>
  
  <p>My code is as follows:</p>
  
  <pre class="lang-py prettyprint-override"><code>def my_net(x, num_classes, phase_train, scope):
      x = tf.layers.conv2d(...)
      x = tf.layers.batch_normalization(x, training=phase_train)
      x = tf.nn.relu(x) 
      x = tf.layers.max_pooling2d(...)
  
      # some other staffs
      ...
  
      # return 
      return x
  
  def train():
      phase_train = tf.placeholder(tf.bool, name='phase_train')
      image_node = tf.placeholder(tf.float32, shape=[batch_size, HEIGHT, WIDTH, 3])
      images, labels = data_loader(train_set)
      val_images, val_labels = data_loader(validation_set)
      prediction_op = my_net(image_node, num_classes=2,phase_train=phase_train, scope='Branch1')
  
      loss_op = loss(...)
      # some other staffs
      optimizer = tf.train.AdamOptimizer(base_learning_rate)
      update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
      with tf.control_dependencies(update_ops):
          train_op = optimizer.minimize(loss=total_loss, global_step=global_step)
      sess = ...
      coord = ...
      while not coord.should_stop():
          image_batch, label_batch = sess.run([images, labels])
          _,loss_value= sess.run([train_op,loss_op], feed_dict={image_node:image_batch,label_node:label_batch,phase_train:True})
  
          step = step+1
  
          if step==NUM_TRAIN_SAMPLES:
              for _ in range(NUM_VAL_SAMPLES/batch_size):
                  image_batch, label_batch = sess.run([val_images, val_labels])
                  prediction_batch = sess.run([prediction_op], feed_dict={image_node:image_batch,label_node:label_batch,phase_train:False})
              val_accuracy = compute_accuracy(...)
  
  
  def test():
      phase_train = tf.placeholder(tf.bool, name='phase_train')
      image_node = tf.placeholder(tf.float32, shape=[batch_size, HEIGHT, WIDTH, 3])
      test_images, test_labels = data_loader(test_set)
      prediction_op = my_net(image_node, num_classes=2,phase_train=phase_train, scope='Branch1')
  
      # some staff to load the trained weights to the graph
      saver.restore(...)
  
      for _ in range(NUM_TEST_SAMPLES/batch_size):
          image_batch, label_batch = sess.run([test_images, test_labels])
          prediction_batch = sess.run([prediction_op], feed_dict={image_node:image_batch,label_node:label_batch,phase_train:False})
      test_accuracy = compute_accuracy(...)
  </code></pre>
  
  <p>The training seems to work well and the <code>val_accuracy</code> is reasonable (say <code>0.70</code>). The problem is: when I tried to use the trained model to do test (i.e., the <code>test</code> function), if the <code>phase_train</code> is set to <code>False</code>, the <code>test_accuracy</code> is very low (say, <code>0.000270</code>), but when the <code>phase_train</code> is set to <code>True</code>, the <code>test_accuracy</code> seems right (say <code>0.69</code>). </p>
  
  <p>As far as I understood, the <code>phase_train</code> should be <code>False</code> in testing stage, right?
  I'm not sure what the problem is. Do I misunderstand the batch normalization? </p>