title: Add low layers in a Tensorflow model
body: <p>Trying to develop some transfert learning algorithm, I use some trained neural networks and add layers. I am using Tensorflow and python.</p>
  
  <p>It seems quite common to use existing graphs in Tensorflow: you import the graph, for example using metaGraphs, then you set new high layers by adding nodes. For example, I found this code <a href="https://blog.metaflow.fr/tensorflow-saving-restoring-and-mixing-multiple-models-c4c94d5d7125" rel="nofollow noreferrer">here</a> :</p>
  
  <pre><code>vgg_saver = tf.train.import_meta_graph(dir + '/vgg/results/vgg-16.meta')
  # Access the graph
  vgg_graph = tf.get_default_graph()
  
  # Retrieve VGG inputs
  self.x_plh = vgg_graph.get_tensor_by_name('input:0')
  
  # Choose some node
  output_conv =vgg_graph.get_tensor_by_name('conv1_2:0')
  
  # Build further operations
  output_conv_shape = output_conv.get_shape().as_list()
  W1 = tf.get_variable('W1', shape=[1, 1, output_conv_shape[3], 32],initializer=tf.random_normal_initializer(stddev=1e-1))
  b1 = tf.get_variable('b1', shape=[32], initializer=tf.constant_initializer(0.1))
  z1 = tf.nn.conv2d(output_conv, W1, strides=[1, 1, 1, 1], padding='SAME') + b1
  a = tf.nn.relu(z1)
  </code></pre>
  
  <p>Then in the training, you would use your layers plus all those below. You can also freeze some layers, import trained variables during the session, etc.</p>
  
  <p>However, in my approach I need to add new low layers between the input and the first layer, and use my layers plus the ones above. Therefore I can't just add nodes at the bottom of the graph : I need to insert nodes right after the input.</p>
  
  <p>Until now I have found no convenient way to do that with tensorflow. Have you any idea ? Or is it just impossible ?</p>
  
  <p>Thanks in advance.</p>