title: opt.apply_gradients() returns 'Trying to update a Tensor' error
body: <p>my problem is the following: Given a deep learning model and a loss function, I would like to evaluate the gradient of the loss function with respect to the input of the model and then to update the input accordingly.</p>
  <p>The way I tried to solve this problem is very similar to the one of this tutorial, &quot;<a href="https://keras.io/guides/writing_a_training_loop_from_scratch/" rel="nofollow noreferrer">Writing a training loop from scratch</a>&quot;; I had to change the variables with respect to which the gradient is evaluated and then <code>.apply_gradients()</code> is applied.</p>
  <p>Here comes my code:</p>
  <pre><code>target = 0.5 * np.ones(data1.shape)
  
  optimizer = tf.keras.optimizers.SGD(learning_rate=1e-3)
  
  epochs = 180
  
  for epoch in range(epochs):
    with tf.GradientTape() as tape:
      data1 = tf.convert_to_tensor(data1)
      tape.watch(data1)
      logits = pred_block(data1, training=False)
      loss_value = tf.keras.losses.MSE(target, logits)
    grads = tape.gradient(loss_value, data1)
    optimizer.apply_gradients(zip(grads, data1))
  </code></pre>
  <p>Inside the <code>GradientTape()</code> scope <code>data1</code> is converted to a tensor and then is watched by tape because otherwise I cannot take the gradient with respect to <code>data1</code>.</p>
  <p>However, I get the following error:</p>
  <pre><code>/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py in apply_grad_to_update_var(var, grad)
      592       &quot;&quot;&quot;Apply gradient to variable.&quot;&quot;&quot;
      593       if isinstance(var, ops.Tensor):
  --&gt; 594         raise NotImplementedError(&quot;Trying to update a Tensor &quot;, var)
      595 
      596       apply_kwargs = {}
  
  NotImplementedError: ('Trying to update a Tensor ', &lt;tf.Tensor: shape=(5,), dtype=float64, numpy=array([0.30344828, 0.30344828, 0.30344828, 0.        , 0.        ])&gt;)
  </code></pre>
  <p>The tensor which it is reffering to is <code>data1</code>, hence I tried to convert it to a <code>numpy.array</code>. The last row of the code is modified as:</p>
  <pre><code>optimizer.apply_gradients(zip(grads, np.array(data1)))
  </code></pre>
  <p>But I get the following error:</p>
  <pre><code>/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py in _prepare(self, var_list)
      782         var_devices = var._devices   # pylint: disable=protected-access
      783       else:
  --&gt; 784         var_devices = [var.device]
      785       var_dtype = var.dtype.base_dtype
      786       for var_device in var_devices:
  
  AttributeError: 'numpy.ndarray' object has no attribute 'device'
  </code></pre>
  <p>The very same happens if I convert also <code>grads</code>.</p>
  <p>I thought it may be related to the shape of <code>data1</code>, namely <code>(5756,5)</code>, because the error gives <code>NotImplementedError: ('Trying to .... shape = (5,))</code>, so I tried to add a dimension to <code>data1</code>. Before opening the <code>for</code> loop:</p>
  <pre><code>data1 = data1[:, np.newaxis, :] 
  
  for epoch in range(epochs):
      ...
  </code></pre>
  <p>But the same error message kicks in:</p>
  <pre><code>/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py in apply_grad_to_update_var(var, grad)
      592       &quot;&quot;&quot;Apply gradient to variable.&quot;&quot;&quot;
      593       if isinstance(var, ops.Tensor):
  --&gt; 594         raise NotImplementedError(&quot;Trying to update a Tensor &quot;, var)
      595 
      596       apply_kwargs = {}
  
  NotImplementedError: ('Trying to update a Tensor ', &lt;tf.Tensor: shape=(1, 5), dtype=float64, numpy=array([[0.30344828, 0.30344828, 0.30344828, 0.        , 0.        ]])&gt;)
  </code></pre>
  <p>Even changing the <code>training</code> flag from <code>False</code> to <code>True</code> gives the same error.</p>
  <p>I also tried to change the last two lines of the code as follows:</p>
  <pre><code>  grads = tape.gradient(loss_value, data1)
    grads = tf.distribute.get_replica_context().all_reduce('sum', grads)
    optimizer.apply_gradients(zip(grads, data1), 
                              experimental_aggregate_gradients=False) 
  </code></pre>
  <p>Again, the very same error message kicks in:</p>
  <pre><code>/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py in apply_grad_to_update_var(var, grad)
      592       &quot;&quot;&quot;Apply gradient to variable.&quot;&quot;&quot;
      593       if isinstance(var, ops.Tensor):
  --&gt; 594         raise NotImplementedError(&quot;Trying to update a Tensor &quot;, var)
      595 
      596       apply_kwargs = {}
  
  NotImplementedError: ('Trying to update a Tensor ', &lt;tf.Tensor: shape=(5,), dtype=float64, numpy=array([0.30344828, 0.30344828, 0.30344828, 0.        , 0.        ])&gt;)
  </code></pre>
  <p>Is there something wrong that I do?</p>
  <p>Thanks!</p>