title: Deep copy of tensor in tensorflow python
body: <p>In some of my code, I have created a neural network using tensorflow and have access to a tensor representing that network's output. I want to make a copy of this tensor so that even if I train the neural network more, I can access the original value of the tensor.</p>
  
  <p>Following other answers and tensorflow documentation, I have tried the tf.identity() function, but it does not seem to be doing what I need. Some other links suggested the use of tf.tile(), but this did not help either. I do not wish to use sess.run(), evaluate the tensor, and store it elsewhere.</p>
  
  <p>Here is a toy example that describes what I need to do:</p>
  
  <pre class="lang-py prettyprint-override"><code>import tensorflow as tf
  import numpy as np
  
  t1 = tf.placeholder(tf.float32, [None, 1])
  t2 = tf.layers.dense(t1, 1, activation=tf.nn.relu)
  expected_out = tf.placeholder(tf.float32, [None, 1])
  
  loss = tf.reduce_mean(tf.square(expected_out - t2))
  train_op = tf.train.AdamOptimizer(1e-4).minimize(loss)
  
  sess = tf.Session()
  
  sess.run(tf.global_variables_initializer())
  
  print(sess.run(t2, feed_dict={t1: np.array([1]).reshape(-1,1)}))
  t3 = tf.identity(t2) # Need to make copy here
  print(sess.run(t3, feed_dict={t1: np.array([1]).reshape(-1,1)}))
  
  print("\nTraining \n")
  
  for i in range(1000):
      sess.run(train_op, feed_dict={t1: np.array([1]).reshape(-1,1), expected_out: np.array([1]).reshape(-1,1)})
  
  print(sess.run(t2, feed_dict={t1: np.array([1]).reshape(-1,1)}))
  print(sess.run(t3, feed_dict={t1: np.array([1]).reshape(-1,1)}))
  </code></pre>
  
  <p>The result of the above code is that <code>t2</code> and <code>t3</code> have the same value. </p>
  
  <pre><code>[[1.5078927]]
  [[1.5078927]]
  
  Training
  
  [[1.3262703]]
  [[1.3262703]]
  </code></pre>
  
  <p>What I want is for <code>t3</code> to keep its value from being copied.</p>
  
  <pre><code>[[1.5078927]]
  [[1.5078927]]
  
  Training
  
  [[1.3262703]]
  [[1.5078927]]
  </code></pre>
  
  <p>Thanks in advance for your help.</p>