title: Visualizing embeddings on Tensorboard using the output of a preloaded network
body: <p>I was wondering how to visualize the embeddings that are from a preloaded network in Tensorboard. I'm using <a href="https://arxiv.org/pdf/1503.03832.pdf" rel="nofollow noreferrer">FaceNet</a> to create the embeddings for faces, I have already created the <code>sprite.png</code> and <code>labels.tsv</code> files. As for loading the network and setting up for Tensorboard this is what I have done so far:</p>
  
  <p><strong>1. Load the embedding layer</strong></p>
  
  <pre><code>meta_file, ckpt_file = facenet.get_model_filenames(MODEL_DIR)
  with tf.Graph().as_default():
      with tf.Session().as_default() as sess:
          # load the network 
          model_dir_exp = os.path.expanduser(MODEL_DIR)
          saver = tf.train.import_meta_graph(os.path.join(model_dir_exp, meta_file))
          saver.restore(tf.get_default_session(), os.path.join(model_dir_exp, ckpt_file))
  
          # setup the lambda function needed to get the embeddings
          images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
          embeddings = tf.get_default_graph().get_tensor_by_name("embeddings:0")
          phase_train_placeholder = tf.get_default_graph().get_tensor_by_name("phase_train:0")
          find_embeddings = lambda img : sess.run(embeddings, feed_dict = {images_placeholder : img, phase_train_placeholder : False})
  </code></pre>
  
  <p><strong>2. Find the embeddings</strong></p>
  
  <pre><code>face_embeddings = np.zeros((n_images,128))
  face_embeddings = []
  for i in range(n_batches):
      start = i * batch_size
      end = min((i + 1) * batch_size, n_images)
  
      # Get the embeddings
      face_embeddings[start:end, :] = find_embeddings(face_images[start:end])
  </code></pre>
  
  <p><strong>3. Setup Tensorboard</strong></p>
  
  <pre><code>from tensorflow.contrib.tensorboard.plugins import projector
  
  embedding = tf.Variable(tf.zeros([33, 128]), name = "embedding")
  config = projector.ProjectorConfig()
  embedding_config = config.embeddings.add()
  embedding_config.tensor_name = embedding.name
  embedding_config.metadata_path = os.path.join(MODEL_DIR, 'labels.tsv')
  embedding_config.sprite.image_path = os.path.join(MODEL_DIR,'sprite.png')
  embedding_config.sprite.single_image_dim.extend([160, 160])
  
  writer = tf.summary.FileWriter(MODEL_DIR)
  projector.visualize_embeddings(writer, config)
  </code></pre>
  
  <p>Although when I load this in Tensorboard it says that it can't find the data. I've looked at the <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tensorboard/README.md#my-tensorboard-isnt-showing-any-data-whats-wrong" rel="nofollow noreferrer">FAQ</a> and when I run <code>find MODEL_DIR | grep tfevents</code> nothing shows up so I'm guessing that this is the problem. I've looked at the <a href="https://www.youtube.com/watch?v=eBbEDRsCmv4&amp;t=885s" rel="nofollow noreferrer">MNIST Tutorial</a> and it seems that they have checkpoints during the training step although I don't have it as I'm using a pre-trained model. Any ideas as how I would make Tensorboard show the embeddings in this case?</p>