title: Save deep network and change its central layer in Tensorflow
body: <p>I built an autoencoder in TensorFlow. It has 5 hidden layers. I trained the network and I now wish to save it on some external file.</p>
  
  <p>Later, I would like to load the autoencoder back and modify its central layer. Then I would like to run the autoencoder on some input data.</p>
  
  <p>Here my code for the autoencoder. I have inserted some <code>saver</code> lines where I think they may help me save the model. However, I am not sure how to reload the saved model and most importantly how to modify its central layer.</p>
  
  <pre><code>input = ### some data
  output = input
  
  tf.reset_default_graph()
  
  num_inputs=501    
  num_hid1=250
  num_hid2=100
  num_hid3=50
  num_hid4=num_hid2
  num_hid5=num_hid1
  num_output=num_inputs
  lr=0.01
  actf=tf.nn.tanh
  
  X=tf.placeholder(tf.float32,shape=[None,num_inputs])
  initializer=tf.variance_scaling_initializer()
  
  w1=tf.Variable(initializer([num_inputs,num_hid1]),dtype=tf.float32)
  w2=tf.Variable(initializer([num_hid1,num_hid2]),dtype=tf.float32)
  w3=tf.Variable(initializer([num_hid2,num_hid3]),dtype=tf.float32)
  w4=tf.Variable(initializer([num_hid3,num_hid4]),dtype=tf.float32)
  w5=tf.Variable(initializer([num_hid4,num_hid5]),dtype=tf.float32)
  w6=tf.Variable(initializer([num_hid5,num_output]),dtype=tf.float32)
  
  b1=tf.Variable(tf.zeros(num_hid1))
  b2=tf.Variable(tf.zeros(num_hid2))
  b3=tf.Variable(tf.zeros(num_hid3))
  b4=tf.Variable(tf.zeros(num_hid4))
  b5=tf.Variable(tf.zeros(num_hid5))
  b6=tf.Variable(tf.zeros(num_output))
  
  hid_layer1=actf(tf.matmul(X,w1)+b1)
  hid_layer2=actf(tf.matmul(hid_layer1,w2)+b2)
  hid_layer3=actf(tf.matmul(hid_layer2,w3)+b3)
  hid_layer4=actf(tf.matmul(hid_layer3,w4)+b4)
  hid_layer5=actf(tf.matmul(hid_layer4,w5)+b5)
  output_layer=tf.matmul(hid_layer5,w6)+b6
  
  loss=tf.reduce_mean(tf.square(output_layer-X))
  
  optimizer=tf.train.AdamOptimizer(lr)
  train=optimizer.minimize(loss)
  
  init=tf.global_variables_initializer()
  
  num_epoch=100000
  batch_size=150
  
  saver = tf.train.Saver()
  with tf.Session() as sess:
      sess.run(init)
      for epoch in range(num_epoch):
  
          sess.run(train,feed_dict={X:input})
  
          train_loss=loss.eval(feed_dict={X:input})
          print("epoch {} loss {}".format(epoch,train_loss))
  
  
      results=output_layer.eval(feed_dict={X:input})
      saver.save(sess, 'my_test_model')
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>In reply to @mujjiga 's answer, in fact what I have to is to chop off the encoder part of this autoencoder. And then use the remaining decoder to decode a new set of <code>hid_layer3</code> features.</p>