title: Tensorflow: How do the gradients get incorporated into the trainable variables?
body: <p>I am fairly new to TensorFlow. I am struggeling to understand how TF incorporates the gradients into the variables for LSTMs (when using a gradient decent optimizer). </p>
  
  <p>When I call <code>tf.trainable_variables()</code> during the first iteration and <code>tf.gradients()</code>
   during the second iteration I should be able to calculate the <code>tf.trainable_variables()</code> of the second iteration right (by multiplying the gradient with the learning rate)? </p>
  
  <p>As I understood the theory an updated weight is equal to the pre-updated weight minus the gradient of that weight (wrt the loss) times the learning rate. However when I do this for a BasicLSTM cell with a DynamicRNN this does not seem the case.</p>