title: Why the 6 in relu6?
body: <p>I've hacked a deep feed forward NN from scratch in R, and it seems more stable with "hard sigmoid" activations - max(0,min(1,x)) - than ReLU. Trying to port it to TensorFlow, and noticed that they don't have this activation function built in, only relu6, which uses an upper cutoff at 6. Is there a reason for this? 
  (I realize that you could do relu6(x*6)/6, but if the TF guys put the 6 there for a good reason, I'd like to know.)
  Also, I'd like to know if others have explosion problems with ReLU in feed forward nets (I'm aware of RNN issues).</p>