title: Update only part of the word embedding matrix in Tensorflow
body: <p>Assuming that I want to update a pre-trained word-embedding matrix during training, is there a way to update only a subset of the word embedding matrix?</p>
  
  <p>I have looked into the Tensorflow API page and found this:</p>
  
  <pre class="lang-py prettyprint-override"><code># Create an optimizer.
  opt = GradientDescentOptimizer(learning_rate=0.1)
  
  # Compute the gradients for a list of variables.
  grads_and_vars = opt.compute_gradients(loss, &lt;list of variables&gt;)
  
  # grads_and_vars is a list of tuples (gradient, variable).  Do whatever you
  # need to the 'gradient' part, for example cap them, etc.
  capped_grads_and_vars = [(MyCapper(gv[0]), gv[1])) for gv in grads_and_vars]
  
  # Ask the optimizer to apply the capped gradients.
  opt.apply_gradients(capped_grads_and_vars)
  </code></pre>
  
  <p>However how do I apply that to the word-embedding matrix. Suppose I do:</p>
  
  <pre class="lang-py prettyprint-override"><code>word_emb = tf.Variable(0.2 * tf.random_uniform([syn0.shape[0],s['es']], minval=-1.0, maxval=1.0, dtype=tf.float32),name='word_emb',trainable=False)
  
  gather_emb = tf.gather(word_emb,indices) #assuming that I pass some indices as placeholder through feed_dict
  
  opt = tf.train.AdamOptimizer(1e-4)
  grad = opt.compute_gradients(loss,gather_emb)
  </code></pre>
  
  <p>How do I then use <code>opt.apply_gradients</code> and <code>tf.scatter_update</code> to update the original embeddign matrix? (Also, tensorflow throws an error if the second argument of <code>compute_gradient</code> is not a <code>tf.Variable</code>)</p>