title: Hard limiting / threshold activation function in TensorFlow
body: <p>I'm trying to implement a basic, binary <a href="http://www.scholarpedia.org/article/Hopfield_network" rel="noreferrer">Hopfield Network</a> in TensorFlow 0.9. Unfortunately I'm having a very hard time getting the activation function working. I'm looking to get the very simple <code>If net[i] &lt; 0, output[i] = 0, else output[i] = 1</code> but everything I've tried seems to remove the gradient, i.e. I get the "No gradients provided for any variable" exception when trying to implement the training op. </p>
  
  <p>For example, I tried casting <code>tf.less()</code> to <code>float</code>, I tried doing something along the lines of</p>
  
  <pre><code>tf.maximum(tf.minimum(net, 0) + 1, 0)
  </code></pre>
  
  <p>but I forgot about small decimal values. Finally I did</p>
  
  <pre><code>tf.maximum(tf.floor(tf.minimum(net, 0) + 1), 0)
  </code></pre>
  
  <p>but <code>tf.floor</code> doesn't register gradients. I also tried replacing the floor with a cast to int and then a cast back to float but same deal.</p>
  
  <p>Any suggestions on what I could do? </p>