<p dir="auto">When using <code class="notranslate">tf.identity()</code> it seems like a proper implementation to pass through the input tensor without copying. However, when the input is the value of a variable it might change later, leading to surprising behavior:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="var = tf.Variable(1)
old = tf.identity(var.value())
with tf.control_dependencies([old]):
  with tf.control_dependencies([var.assign_add(1)]):
    new = tf.identity(var.value())

sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run([old, new]))  # Unexpected: [2, 2]"><pre class="notranslate"><span class="pl-s1">var</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Variable</span>(<span class="pl-c1">1</span>)
<span class="pl-s1">old</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">identity</span>(<span class="pl-s1">var</span>.<span class="pl-en">value</span>())
<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">control_dependencies</span>([<span class="pl-s1">old</span>]):
  <span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">control_dependencies</span>([<span class="pl-s1">var</span>.<span class="pl-en">assign_add</span>(<span class="pl-c1">1</span>)]):
    <span class="pl-s1">new</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">identity</span>(<span class="pl-s1">var</span>.<span class="pl-en">value</span>())

<span class="pl-s1">sess</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Session</span>()
<span class="pl-s1">sess</span>.<span class="pl-en">run</span>(<span class="pl-s1">tf</span>.<span class="pl-en">global_variables_initializer</span>())
<span class="pl-en">print</span>(<span class="pl-s1">sess</span>.<span class="pl-en">run</span>([<span class="pl-s1">old</span>, <span class="pl-s1">new</span>]))  <span class="pl-c"># Unexpected: [2, 2]</span></pre></div>
<p dir="auto">I would think this code should be equivalent to the following workaround that uses a second variable to remember the previous value:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="var = tf.Variable(1)
old = tf.Variable(var.initialized_value())
with tf.control_dependencies([old.assign(var.value())]):
  with tf.control_dependencies([var.assign_add(1)]):
    new = tf.identity(var.value())

sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run([old, new]))  # Expected: [1, 2]"><pre class="notranslate"><span class="pl-s1">var</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Variable</span>(<span class="pl-c1">1</span>)
<span class="pl-s1">old</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Variable</span>(<span class="pl-s1">var</span>.<span class="pl-en">initialized_value</span>())
<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">control_dependencies</span>([<span class="pl-s1">old</span>.<span class="pl-en">assign</span>(<span class="pl-s1">var</span>.<span class="pl-en">value</span>())]):
  <span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">control_dependencies</span>([<span class="pl-s1">var</span>.<span class="pl-en">assign_add</span>(<span class="pl-c1">1</span>)]):
    <span class="pl-s1">new</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">identity</span>(<span class="pl-s1">var</span>.<span class="pl-en">value</span>())

<span class="pl-s1">sess</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Session</span>()
<span class="pl-s1">sess</span>.<span class="pl-en">run</span>(<span class="pl-s1">tf</span>.<span class="pl-en">global_variables_initializer</span>())
<span class="pl-en">print</span>(<span class="pl-s1">sess</span>.<span class="pl-en">run</span>([<span class="pl-s1">old</span>, <span class="pl-s1">new</span>]))  <span class="pl-c"># Expected: [1, 2]</span></pre></div>
<p dir="auto">Can we make <code class="notranslate">tf.identity()</code> aware of whether its input is static or varying, to always return the value from the time it's executed?</p>
<p dir="auto">I'm running Tensorflow 0.10.</p>
<p dir="auto">The following code</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import tensorflow as tf

x = tf.Variable(0, dtype=tf.int32)

old_val = tf.identity(x)
with tf.control_dependencies([old_val]):
    new_val = tf.assign(x, x + 1)

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())

    for i in xrange(3):
        print sess.run([old_val, new_val, x])"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">tensorflow</span> <span class="pl-k">as</span> <span class="pl-s1">tf</span>

<span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-v">Variable</span>(<span class="pl-c1">0</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">tf</span>.<span class="pl-s1">int32</span>)

<span class="pl-s1">old_val</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">identity</span>(<span class="pl-s1">x</span>)
<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">control_dependencies</span>([<span class="pl-s1">old_val</span>]):
    <span class="pl-s1">new_val</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">assign</span>(<span class="pl-s1">x</span>, <span class="pl-s1">x</span> <span class="pl-c1">+</span> <span class="pl-c1">1</span>)

<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-v">Session</span>() <span class="pl-k">as</span> <span class="pl-s1">sess</span>:
    <span class="pl-s1">sess</span>.<span class="pl-en">run</span>(<span class="pl-s1">tf</span>.<span class="pl-en">initialize_all_variables</span>())

    <span class="pl-k">for</span> <span class="pl-s1">i</span> <span class="pl-c1">in</span> <span class="pl-en">xrange</span>(<span class="pl-c1">3</span>):
        <span class="pl-k">print</span> <span class="pl-s1">sess</span>.<span class="pl-en">run</span>([<span class="pl-s1">old_val</span>, <span class="pl-s1">new_val</span>, <span class="pl-s1">x</span>])</pre></div>
<p dir="auto">outputs</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[1, 1, 1]
[2, 2, 2]
[3, 3, 3]"><pre class="notranslate"><code class="notranslate">[1, 1, 1]
[2, 2, 2]
[3, 3, 3]
</code></pre></div>
<p dir="auto">From reading the docs on <code class="notranslate">control_dependencies</code> and <code class="notranslate">identity</code> as well as StackOverflow, I expected output</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[0, 1, ?]
[1, 2, ?]
[2, 3, ?]"><pre class="notranslate"><code class="notranslate">[0, 1, ?]
[1, 2, ?]
[2, 3, ?]
</code></pre></div>
<p dir="auto">where <code class="notranslate">?</code> indicates that the variable value is unspecified.</p>
<p dir="auto">Is this a bug? If this is not a bug, what is the correct way to refer to the value of variable before and after assignment in a single graph?</p>