title: FailedPreconditionError while trying to use RMSPropOptimizer on tensorflow
body: <p>I am trying to use the RMSPropOptimizer for minimizing loss. Here's the part of the code that is relevant:</p>
  
  <pre><code>import tensorflow as tf
  
  #build large convnet...
  #...
  
  opt = tf.train.RMSPropOptimizer(learning_rate=0.0025, decay=0.95)
  
  #do stuff to get targets and loss...
  #...
  
  grads_and_vars = opt.compute_gradients(loss)
  capped_grads_and_vars = [(tf.clip_by_value(g, -1, 1), v) for g, v in grads_and_vars]
  opt_op = self.opt.apply_gradients(capped_grads_and_vars)
  
  sess = tf.Session()
  sess.run(tf.initialize_all_variables())
  while(1):
      sess.run(opt_op)
  </code></pre>
  
  <p>Problem is as soon as I run this I get the following error:</p>
  
  <pre><code>W tensorflow/core/common_runtime/executor.cc:1091] 0x10a0bba40 Compute status: Failed precondition: Attempting to use uninitialized value train/output/bias/RMSProp
       [[Node: RMSProp/update_train/output/bias/ApplyRMSProp = ApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/cpu:0"](train/output/bias, train/output/bias/RMSProp, train/output/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, clip_by_value_9)]]
       [[Node: _send_MergeSummary/MergeSummary_0 = _Send[T=DT_STRING, client_terminated=true, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=-6901001318975381332, tensor_name="MergeSummary/MergeSummary:0", _device="/job:localhost/replica:0/task:0/cpu:0"](MergeSummary/MergeSummary)]]
  Traceback (most recent call last):
    File "dqn.py", line 213, in &lt;module&gt;
      result = sess.run(opt_op)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 385, in run
      results = self._do_run(target_list, unique_fetch_targets, feed_dict_string)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 461, in _do_run
      e.code)
  tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value train/output/bias/RMSProp
       [[Node: RMSProp/update_train/output/bias/ApplyRMSProp = ApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/cpu:0"](train/output/bias, train/output/bias/RMSProp, train/output/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, clip_by_value_9)]]
  Caused by op u'RMSProp/update_train/output/bias/ApplyRMSProp', defined at: 
    File "dqn.py", line 159, in qLearnMinibatch
      opt_op = self.opt.apply_gradients(capped_grads_and_vars)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 288, in apply_gradients
      update_ops.append(self._apply_dense(grad, var))
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/training/rmsprop.py", line 103, in _apply_dense
      grad, use_locking=self._use_locking).op
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/training/gen_training_ops.py", line 171, in apply_rms_prop
      grad=grad, use_locking=use_locking, name=name)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 659, in apply_op
      op_def=op_def)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1904, in create_op
      original_op=self._default_original_op, op_def=op_def)
    File "/Users/home/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1083, in __init__
      self._traceback = _extract_stack()
  </code></pre>
  
  <p>Note that I don't get this error If am using the usual GradientDescentOptimizer. I am initializing my variables as you can see above but I don't know what 'train/output/bias/RMSProp' is because I don't create any such variable. I only have 'train/output/bias/' which does get initialized above.</p>
  
  <p>Thanks!</p>