title: Why do tensorflow and keras SimpleRNN layers have a default activation of tanh
body: <p>I want to use a relu activation for my simple RNN in a tensorflow model I am building.  It sits on top of a deep convolutional network.  I am trying to classify a sequence of images.  I noticed that the default activation in both keras and tensorflow source code is tanh for simple RNNs.  Is there a reason for this?  Is there anything wrong with using relu?  It seems like relu would help better with the vanishing gradients.</p>
  
  <p><code>nn = tf.nn.rnn_cell.BasicRNNCell(1024, activation = tf.nn.relu)</code></p>