title: Tensorflow: ValueError: No gradients provided for any variable
body: <p>I am confused as of why the following functions throws the error <code>ValueError: No gradients provided for any variable</code>.</p>
  <pre class="lang-py prettyprint-override"><code>@tf.function
  def train_network(model, optimizer, states, actions, rewards):
      with tf.GradientTape() as tape:
          predictions = model(states)
          indices = tf.stack([tf.range(predictions.shape[0], dtype=tf.int64), actions], axis=1)
          action_rewards = tf.scatter_nd(indices, rewards, predictions.shape)
          loss = tf.keras.losses.MSE(predictions, action_rewards)
      grads = tape.gradient(loss, model.trainable_variables)
      optimizer.apply_gradients(zip(grads, model.trainable_variables))
      return loss
  </code></pre>
  <p>I pass the model and optimizer because they are part of a class-object. But as I found you cannot have a tf.function as a class-function.</p>
  <p>If I try the same thing with a keras fit function it works just fine, eg. <code>self.model.fit(np.array(states_epoch), np.array(rewards_epoch), batch_size=50)</code>.</p>
  <p>I want to run it with a tf function because my training is actually more complex but for this example I slimmed it down as much as possible.</p>
  <p>More info:</p>
  <p>states are float32, actions is a list of integers of actions took, reward is a list of integers of received rewards.</p>
  <p>The network is a simple DNN, optimizer is keras.</p>