title: What happens when setting reuse=True in tensorflow
body: <p>I would like to know what happens in the background when we set the reuse to True in tensorflow. </p>
  
  <p>Basically, when building models in tensorflow for training and testing, I have to create the model first within a method and then call that within a variable scope as follows:</p>
  
  <pre><code>def model(inputs, return_top=True):
      #.... Here I have several conv layers
  
      if return_top:
          output = tf.layers.dense(output, units=8, name='outputs')
  
      return output
  
  with tf.variable_scope('model'):
      output_train = model(inputs_train)
      mse_train = cal_loss(output_train, labels_train) # This is a function that calculates the loss
      train_step = optimize(mse_train)    # This is a function that implements the optimizer
  
  with tf.variable_scope('model', reuse=True):
      output_validation = model(inputs_validation)
      mse_validation = cal_loss(output_validation, labels_validation)
  </code></pre>
  
  <p>When creating models in tensorflow for training and testing, we usually create one model for training; and let's assume that we give it a name "model"; i.e. we created the whole model under a <code>tf.variable_scope("model", reuse=False)</code>; and then we reused the model for testing where we set reuse to True. Thus, we use <code>with tf.variable_scope("model", reuse=True)</code>. Now if I look into tensorboard, I find two copies for the whole model, one under name "model", and the other under "model_1". Also, I found that "model_1" references "model"; i.e., the weights of "model_1" are taken from "model" (That's my assumption; I would like to know if this is true). Also, I found that the "model" outputs goes into the optimizer, which is not the case with "model_1". I wonder why. In other words, if "model_1" references "model"; and the optimizer modifies the weights of "model"; should it modify the weights of "model_1"?</p>
  
  <p>Any help is much appreciated!!</p>