<p dir="auto">Thank you for submitting a TensorFlow documentation issue. Per our GitHub<br>
policy, we only address code/doc bugs, performance issues, feature requests, and<br>
build/installation issues on GitHub.</p>
<p dir="auto">The TensorFlow docs are open source! To get involved, read the documentation<br>
contributor guide: <a href="https://www.tensorflow.org/community/contribute/docs" rel="nofollow">https://www.tensorflow.org/community/contribute/docs</a></p>
<h2 dir="auto">URL(s) with the issue:</h2>
<p dir="auto">Please provide a link to the documentation entry, for example:<br>
<a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/MyMethod" rel="nofollow">https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/MyMethod</a></p>
<h2 dir="auto">Description of issue (what needs changing):</h2>
<h3 dir="auto">Clear description</h3>
<p dir="auto">For example, why should someone use this method? How is it useful?</p>
<h3 dir="auto">Correct links</h3>
<p dir="auto">Is the link to the source code correct?</p>
<h3 dir="auto">Parameters defined</h3>
<p dir="auto">Are all parameters defined and formatted correctly?</p>
<h3 dir="auto">Returns defined</h3>
<p dir="auto">Are return values defined?</p>
<h3 dir="auto">Raises listed and defined</h3>
<p dir="auto">Are the errors defined? For example,<br>
<a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/feature_column/categorical_column_with_vocabulary_file#raises" rel="nofollow">https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/feature_column/categorical_column_with_vocabulary_file#raises</a></p>
<h3 dir="auto">Usage example</h3>
<p dir="auto">Is there a usage example?</p>
<h3 dir="auto">Request visuals, if applicable</h3>
<p dir="auto">Are there currently visuals? If not, will it clarify the content?</p>
<h3 dir="auto">Submit a pull request?</h3>
<p dir="auto">Are you planning to also submit a pull request to fix the issue? See the docs<br>
contributor guide: <a href="https://www.tensorflow.org/community/contribute/docs" rel="nofollow">https://www.tensorflow.org/community/contribute/docs</a> and the<br>
docs style guide: <a href="https://www.tensorflow.org/community/contribute/docs_style" rel="nofollow">https://www.tensorflow.org/community/contribute/docs_style</a></p>
<p dir="auto">I am trying to restore the trained model and retrain it with some additional operations.</p>
<p dir="auto">I have 2 python files, lets say</p>
<p dir="auto">train.py - To train and save the model<br>
retrain.py - Load the trained model, add new elements in graph and retrain</p>
<p dir="auto"><strong>train.py</strong></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def train():
    # 1 NN
    Xinp1 = tf.placeholder(&quot;float&quot;, [None, 2], name=&quot;Xinp1&quot;)
    Xhidden1 = tf.layers.dense(Xinp1, units=16 , 
                kernel_initializer=tf.initializers.he_uniform(), 
                activation=tf.nn.relu, name=&quot;X_hidden1&quot;)
    Xout = tf.layers.dense(X_hidden5, units=1, 
 kernel_initializer=tf.initializers.he_uniform(),activation=tf.nn.sigmoid, name=&quot;X_out&quot;)

    Xout1 = tf.identity(Xout, name=&quot;Xout1&quot;)

    #2 NN
    Xinp2 = tf.placeholder(&quot;float&quot;, [None, 2], name=&quot;Xinp2&quot;)
    Xhidden2 = tf.layers.dense(Xinp2, units=16 , 
                kernel_initializer=tf.initializers.he_uniform(), 
                activation=tf.nn.relu, name=&quot;X_hidden2&quot;)
    Xout = tf.layers.dense(X_hidden2, units=1, 
kernel_initializer=tf.initializers.he_uniform(),activation=tf.nn.sigmoid, name=&quot;X_out&quot;)

    Xout2 = tf.identity(Xout, name=&quot;Xout2&quot;)

    Xout1_label = tf.placeholder(&quot;float&quot;, [None,1], name=&quot;Xout1_label&quot;)
    Xout2_label = tf.placeholder(&quot;float&quot;, [None,1],name=&quot;Xout2_label&quot;)


    learning_rate = 1e-2
    # Define loss and optimizer
    loss_op1 = tf.losses.absolute_difference(Xout1_label, Xout1)
    loss_op2 = tf.losses.absolute_difference(Xout2_label, Xout2)



    # debug gradients
    trainables = tf.trainable_variables()
    print (&quot;trainables&quot;, trainables)
    optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate, epsilon=0.1)

    train_op1 = optimizer.minimize(loss_op1)
    train_op2 = optimizer.minimize(loss_op2)

    with tf.Session() as sess:
          sess.run(tf.global_variables_initializer())
          saver = tf.train.Saver()
          for _ in range(100):
               _, c1, summary = sess.run([train_op1, loss_op1, merged_summary_op], feed_dict={
            Xinp1: X1,
            Xinp2: X2,
            Xout1_label: X1label,
            Xout2_label: X2label
            })    
               _, c2, summary = sess.run([train_op2, loss_op2, merged_summary_op], feed_dict={
            Xinp1: X1,
            Xinp2: X2,
            Xout1_label: X1label,
            Xout2_label: X2label
            })        
          saver.save(sess, 'Model/trained.ckpt')
          sess.close()"><pre class="notranslate"><code class="notranslate">def train():
    # 1 NN
    Xinp1 = tf.placeholder("float", [None, 2], name="Xinp1")
    Xhidden1 = tf.layers.dense(Xinp1, units=16 , 
                kernel_initializer=tf.initializers.he_uniform(), 
                activation=tf.nn.relu, name="X_hidden1")
    Xout = tf.layers.dense(X_hidden5, units=1, 
 kernel_initializer=tf.initializers.he_uniform(),activation=tf.nn.sigmoid, name="X_out")

    Xout1 = tf.identity(Xout, name="Xout1")

    #2 NN
    Xinp2 = tf.placeholder("float", [None, 2], name="Xinp2")
    Xhidden2 = tf.layers.dense(Xinp2, units=16 , 
                kernel_initializer=tf.initializers.he_uniform(), 
                activation=tf.nn.relu, name="X_hidden2")
    Xout = tf.layers.dense(X_hidden2, units=1, 
kernel_initializer=tf.initializers.he_uniform(),activation=tf.nn.sigmoid, name="X_out")

    Xout2 = tf.identity(Xout, name="Xout2")

    Xout1_label = tf.placeholder("float", [None,1], name="Xout1_label")
    Xout2_label = tf.placeholder("float", [None,1],name="Xout2_label")


    learning_rate = 1e-2
    # Define loss and optimizer
    loss_op1 = tf.losses.absolute_difference(Xout1_label, Xout1)
    loss_op2 = tf.losses.absolute_difference(Xout2_label, Xout2)



    # debug gradients
    trainables = tf.trainable_variables()
    print ("trainables", trainables)
    optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate, epsilon=0.1)

    train_op1 = optimizer.minimize(loss_op1)
    train_op2 = optimizer.minimize(loss_op2)

    with tf.Session() as sess:
          sess.run(tf.global_variables_initializer())
          saver = tf.train.Saver()
          for _ in range(100):
               _, c1, summary = sess.run([train_op1, loss_op1, merged_summary_op], feed_dict={
            Xinp1: X1,
            Xinp2: X2,
            Xout1_label: X1label,
            Xout2_label: X2label
            })    
               _, c2, summary = sess.run([train_op2, loss_op2, merged_summary_op], feed_dict={
            Xinp1: X1,
            Xinp2: X2,
            Xout1_label: X1label,
            Xout2_label: X2label
            })        
          saver.save(sess, 'Model/trained.ckpt')
          sess.close()
</code></pre></div>
<p dir="auto">As an output, I got following files</p>
<ol dir="auto">
<li>checkpoint</li>
<li>trained.ckpt.data-00000-of-00001</li>
<li>trained.ckpt.index</li>
<li>trained.ckpt.meta</li>
</ol>
<p dir="auto"><strong>retrain.py</strong></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def retrain():
     with tf.Session() as sess:
           saver = tf.train.import_meta_graph('Model/trained.ckpt.meta')
           saver.restore(sess, 'Model/trained.ckpt')
           graph = tf.get_default_graph()
           Xinp1 = graph.get_tensor_by_name('Xinp1:0')
           Xout1 = graph.get_tensor_by_name('Xout1:0')
           Xinp2 = graph.get_tensor_by_name('Xinp2:0')
           Xout2 = graph.get_tensor_by_name('Xout2:0') 

           # I want to add some additional nodes
           T1 = tf.placeholder(&quot;float&quot;, [None, 1], name=&quot;T1&quot;)
           T2 = tf.placeholder(&quot;float&quot;, [None, 1], name=&quot;T2&quot;)
           Add1 = tf.add(tf.multiply(Xout1, tf.subtract(T1, T2)), T2, name=&quot;Add1_out&quot;)

           T3 = tf.placeholder(&quot;float&quot;, [None, 1], name=&quot;T3&quot;)
           Add2 = tf.multiply(tf.multiply(T3,tf.subtract(Add1, 300)),tf.multiply(radial_length,0.000001), name=&quot;Add2_out&quot;)

           Addlabel = tf.placeholder(&quot;float&quot;, [None, 1], name=&quot;Addlabel&quot;)

           loss_op = tf.losses.mean_squared_error(Addlabel, Add2)

           # debug gradients
           trainables = tf.trainable_variables()
           print (&quot;trainables&quot;, trainables)
           optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate, epsilon=0.1)
           train_op = optimizer.minimize(loss_op)

           sess.run(tf.global_variables_initializer())
           #training starts
           # Here I except weights of 1 NN and 2 NN are learned during the training
           for _ in range(100):
               _, c, summary = sess.run([train_op, loss_op, merged_summary_op], feed_dict={
               Xinp1 : NewX1,
               Xinp2 : NewX2,
               T1 : T1inp,
               T2 : T2inp,
               T3 : T3inp,
               Addlabel : Addtarget               
                }) "><pre class="notranslate"><code class="notranslate">def retrain():
     with tf.Session() as sess:
           saver = tf.train.import_meta_graph('Model/trained.ckpt.meta')
           saver.restore(sess, 'Model/trained.ckpt')
           graph = tf.get_default_graph()
           Xinp1 = graph.get_tensor_by_name('Xinp1:0')
           Xout1 = graph.get_tensor_by_name('Xout1:0')
           Xinp2 = graph.get_tensor_by_name('Xinp2:0')
           Xout2 = graph.get_tensor_by_name('Xout2:0') 

           # I want to add some additional nodes
           T1 = tf.placeholder("float", [None, 1], name="T1")
           T2 = tf.placeholder("float", [None, 1], name="T2")
           Add1 = tf.add(tf.multiply(Xout1, tf.subtract(T1, T2)), T2, name="Add1_out")

           T3 = tf.placeholder("float", [None, 1], name="T3")
           Add2 = tf.multiply(tf.multiply(T3,tf.subtract(Add1, 300)),tf.multiply(radial_length,0.000001), name="Add2_out")

           Addlabel = tf.placeholder("float", [None, 1], name="Addlabel")

           loss_op = tf.losses.mean_squared_error(Addlabel, Add2)

           # debug gradients
           trainables = tf.trainable_variables()
           print ("trainables", trainables)
           optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate, epsilon=0.1)
           train_op = optimizer.minimize(loss_op)

           sess.run(tf.global_variables_initializer())
           #training starts
           # Here I except weights of 1 NN and 2 NN are learned during the training
           for _ in range(100):
               _, c, summary = sess.run([train_op, loss_op, merged_summary_op], feed_dict={
               Xinp1 : NewX1,
               Xinp2 : NewX2,
               T1 : T1inp,
               T2 : T2inp,
               T3 : T3inp,
               Addlabel : Addtarget               
                }) 
</code></pre></div>
<p dir="auto">I am expecting the retrain.py to adjust the weights associated with 1 NN and 2 NN during the training.</p>
<p dir="auto">But instead while running the retrain.py, I am getting the following error</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Traceback (most recent call last):
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py&quot;, line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Duplicate node name in graph: 'X_hidden1/kernel/Adam'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File &quot;/home/itmsec/Documents/tipclearance/src/TTG_tensorflowv14.py&quot;, line 493, in &lt;module&gt;
    restore_and_retrain(BDD)
  File &quot;/home/itmsec/Documents/tipclearance/src/TTG_tensorflowv14.py&quot;, line 244, in restore_and_retrain
    train_op = optimizer.minimize(loss_op)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py&quot;, line 413, in minimize
    name=name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py&quot;, line 595, in apply_gradients
    self._create_slots(var_list)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/adam.py&quot;, line 135, in _create_slots
    self._zeros_slot(v, &quot;m&quot;, self._name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py&quot;, line 1153, in _zeros_slot
    new_slot_variable = slot_creator.create_zeros_slot(var, op_name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py&quot;, line 183, in create_zeros_slot
    colocate_with_primary=colocate_with_primary)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py&quot;, line 157, in create_slot_with_initializer
    dtype)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py&quot;, line 65, in _create_slot_var
    validate_shape=validate_shape)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 1479, in get_variable
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 1220, in get_variable
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 547, in get_variable
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 499, in _true_getter
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 911, in _get_single_variable
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 213, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 176, in _variable_v1_call
    aggregation=aggregation)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 155, in &lt;lambda&gt;
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py&quot;, line 2495, in default_variable_creator
    expected_shape=expected_shape, import_scope=import_scope)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 217, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 1395, in __init__
    constraint=constraint)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py&quot;, line 1509, in _init_from_args
    name=name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/state_ops.py&quot;, line 79, in variable_op_v2
    shared_name=shared_name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_state_ops.py&quot;, line 1425, in variable_v2
    shared_name=shared_name, name=name)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py&quot;, line 788, in _apply_op_helper
    op_def=op_def)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py&quot;, line 507, in new_func
    return func(*args, **kwargs)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py&quot;, line 3300, in create_op
    op_def=op_def)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py&quot;, line 1823, in __init__
    control_input_ops)
  File &quot;/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py&quot;, line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Duplicate node name in graph: 'X_hidden1/kernel/Adam'"><pre class="notranslate"><code class="notranslate">Traceback (most recent call last):
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Duplicate node name in graph: 'X_hidden1/kernel/Adam'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/itmsec/Documents/tipclearance/src/TTG_tensorflowv14.py", line 493, in &lt;module&gt;
    restore_and_retrain(BDD)
  File "/home/itmsec/Documents/tipclearance/src/TTG_tensorflowv14.py", line 244, in restore_and_retrain
    train_op = optimizer.minimize(loss_op)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 413, in minimize
    name=name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 595, in apply_gradients
    self._create_slots(var_list)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/adam.py", line 135, in _create_slots
    self._zeros_slot(v, "m", self._name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 1153, in _zeros_slot
    new_slot_variable = slot_creator.create_zeros_slot(var, op_name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py", line 183, in create_zeros_slot
    colocate_with_primary=colocate_with_primary)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py", line 157, in create_slot_with_initializer
    dtype)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/slot_creator.py", line 65, in _create_slot_var
    validate_shape=validate_shape)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1479, in get_variable
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1220, in get_variable
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 547, in get_variable
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 499, in _true_getter
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 911, in _get_single_variable
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 213, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 176, in _variable_v1_call
    aggregation=aggregation)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 155, in &lt;lambda&gt;
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2495, in default_variable_creator
    expected_shape=expected_shape, import_scope=import_scope)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 217, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1395, in __init__
    constraint=constraint)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1509, in _init_from_args
    name=name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/state_ops.py", line 79, in variable_op_v2
    shared_name=shared_name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_state_ops.py", line 1425, in variable_v2
    shared_name=shared_name, name=name)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/home/itmsec/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Duplicate node name in graph: 'X_hidden1/kernel/Adam'
</code></pre></div>