title: Persist GradientTape after Construction
body: <p>Is it possible to set persistence of a <code>GradientTape</code> after its construction?</p>
  <p>The use case is when I have code that I don't control pass a non-persistent tape to me, with which I must take two gradients with dependencies between them.</p>
  <pre class="lang-py prettyprint-override"><code># I don't control this
  with tf.GradientTape() as tape:
      y = f(x)
      z = g(y)
  
  # I control this
  dzdx = tape.gradient(z, x)
  result = tape.gradient(z, y, output_gradients=dzdx)  # not persistent
  </code></pre>
  <p>I've pondered the possibility of simply setting <code>tape._persistent</code> before calling <code>gradient</code>, but the persistence gets <a href="https://github.com/tensorflow/tensorflow/blob/v2.4.1/tensorflow/python/eager/backprop.py#L856" rel="nofollow noreferrer">passed all the way</a> to C++ code at construction, which probably won't like being inconsistent with the Python code</p>
  <pre class="lang-py prettyprint-override"><code>def push_new_tape(persistent=False, watch_accessed_variables=True):
    &quot;&quot;&quot;Pushes a new tape onto the tape stack.&quot;&quot;&quot;
    tape = pywrap_tfe.TFE_Py_TapeSetNew(persistent, watch_accessed_variables)
    return Tape(tape)
  </code></pre>
  <p>Which ultimately <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/c/eager/tape.h#L125" rel="nofollow noreferrer">ends up at</a></p>
  <pre class="lang-cpp prettyprint-override"><code>template &lt;typename Gradient, typename BackwardFunction, typename TapeTensor&gt;
  class GradientTape {
   public:
    // If `persistent` is true, GradientTape will not eagerly delete backward
    // functions (and hence the tensors they keep alive). Instead, everything
    // is deleted in ~GradientTape. Persistent GradientTapes are useful when
    // users want to compute multiple gradients over the same tape.
    explicit GradientTape(bool persistent) : persistent_(persistent) {}
    ~GradientTape() {
      for (const auto&amp; pair : op_tape_) {
        pair.second.backward_function_deleter(pair.second.backward_function);
      }
    }
  
    // ...
  
    bool IsPersistent() const { return persistent_; }
  </code></pre>
  <p>Where <code>IsPersistent</code> is the only public interface to <code>persistent_</code>. I can't tell how exactly <code>persistent_</code> is used, and that sounds like a memory leak in waiting.</p>