title: TensorFlow exponential moving average
body: <p>I can't figure out how to get <code>tf.train.ExponentialMovingAverage</code> to work. Following is a simple code to find <code>w</code> in a simple <code>y_ = x * w</code> equation. <code>m</code> is the moving average. Why is the code returning <code>None</code> for <code>m</code>? How can I get it to return the moving average value?</p>
  
  <pre><code>import numpy as np
  import tensorflow as tf
  
  w = tf.Variable(0, dtype=tf.float32)
  ema = tf.train.ExponentialMovingAverage(decay=0.9)
  m = ema.apply([w])
  
  x = tf.placeholder(tf.float32, [None])
  y = tf.placeholder(tf.float32, [None])
  y_ = tf.multiply(x, w)
  
  with tf.control_dependencies([m]):
      loss = tf.reduce_sum(tf.square(tf.subtract(y, y_)))
      optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
      train = optimizer.minimize(loss)
  
  with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      for i in range(20):
          _, w_, m_ = sess.run([train, w, m], feed_dict={x: [1], y: [10]})
          print(w_, ',', m_)
  </code></pre>
  
  <p>The output is:</p>
  
  <pre><code>0.02 , None
  0.03996 , None
  0.0598801 , None
  0.0797603 , None
  0.0996008 , None
  0.119402 , None
  0.139163 , None
  0.158884 , None
  0.178567 , None
  0.19821 , None
  0.217813 , None
  0.237378 , None
  0.256903 , None
  0.276389 , None
  0.295836 , None
  0.315244 , None
  0.334614 , None
  0.353945 , None
  0.373237 , None
  0.39249 , None
  </code></pre>