title: Tensorflow : What is actually tf.nn.dropout output_keep_prob?
body: <p>I am trying to understand concept of <code>output_keep_prob</code>:</p>
  
  <p>So if my example is simple RNN :</p>
  
  <pre><code>    with tf.variable_scope('encoder') as scope:
          cells = rnn.LSTMCell(num_units=500)
          cell = rnn.DropoutWrapper(cell=cells, output_keep_prob=0.5)
  
          model = tf.nn.bidirectional_dynamic_rnn(cell, cell, inputs=embedding_lookup, sequence_length=sequence_le,
                                                  dtype=tf.float32)
  </code></pre>
  
  <p>My confusion is if I am giving <code>output_keep_prob=0.5</code> what actually it means? I know that it makes less prone to overfitting (called regularizing) by adding a dropout. It  randomly turns off activations of neurons during training, ok I got this point but I am confused when I give</p>
  
  <p><code>output_keep_prob=0.5</code> and my no_of_nodes = 500 then 0.5 means it will randomly turn of 50% nodes at each iteration or it means it will keep only those connections which have more or equal probability of 0.5   </p>
  
  <pre><code>keep_layers whose probability =&gt;0.5  
  </code></pre>
  
  <p>or</p>
  
  <pre><code>turn off 50% randomly nodes unit at each iteration ??
  </code></pre>
  
  <p>I tried to understand the concept by this <a href="https://stackoverflow.com/questions/35545798/keep-prob-in-tensorflow-mnist-tutorial">stackoverflow answer</a> but there is also same confusion that what actually 0.5 means? it should drop 50% nodes at each iteration or keep only those nodes which have probability more or equal to 0.5 </p>
  
  <p>if the answer is second <code>keep only those nodes which have probability more or equal to 0.5</code> :</p>
  
  <p>then it means suppose I have given 500 nodes units and only 30 nodes have 0.5 probability so it will turn of rest 470 nodes and will use only 30 nodes for incoming and outgoing connections?</p>
  
  <p>Because  <a href="https://stackoverflow.com/questions/44107208/in-tf-nn-dropout-what-is-the-effect-of-keep-prob-argument">this answer</a> says :</p>
  
  <blockquote>
    <p>Suppose you have 10 units in the layer and set the keep_prob to 0.1,
    Then the activation of 9 randomly chosen units out of 10 will be set
    to 0, and the remaining one will be scaled by a factor of 10. I think
    a more precise description is that you only keep the activation of 10
    percent of the nodes.</p>
  </blockquote>
  
  <p>While other side <a href="https://stackoverflow.com/questions/35545798/keep-prob-in-tensorflow-mnist-tutorial">this answer</a> by @mrry says :</p>
  
  <blockquote>
    <p>it means that each connection between layers (in this case between the
    last densely connected layer and the readout layer) will only be used
    with probability 0.5 when training.</p>
  </blockquote>
  
  <p>can anyone give a clear explanation which one is correct and what actually this value represent in keep_prob?</p>