title: Tensorflow negative sampling
body: <p>I am trying to follow the udacity tutorial on tensorflow where I came across the following two lines for word embedding models:</p>
  
  <pre><code>  # Look up embeddings for inputs.
    embed = tf.nn.embedding_lookup(embeddings, train_dataset)
    # Compute the softmax loss, using a sample of the negative labels each time.
    loss = tf.reduce_mean(tf.nn.sampled_softmax_loss(softmax_weights, softmax_biases, 
                          embed, train_labels, num_sampled, vocabulary_size))
  </code></pre>
  
  <p>Now I understand that the second statement is for sampling negative labels. But the question is how does it know what the negative labels are? All I am providing the second function is the current input and its corresponding labels along with number of labels that I want to (negatively) sample from. Isn't there the risk of sampling from the input set in itself?</p>
  
  <p>This is the full example: <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/5_word2vec.ipynb" rel="noreferrer">https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/5_word2vec.ipynb</a></p>