title: Training RNN on GPU - which tf.keras layer should I use?
body: <p>I am training RNNs, which I built using tf.keras.layers.GRU layers. They are taking a long time to train (>2 hours), so I am going to deploy them to the GPU for training. I am wondering a few things about training on GPU:</p>
  
  <ol>
  <li>What is the difference between <code>tf.keras.layers.CuDNNGRU</code> and <code>tf.keras.layers.GRU</code> (and also <code>tf.keras.layers.LSTM</code> vs. <code>tf.keras.layers.CuDNNLSTM</code>)? I understand from <a href="https://stackoverflow.com/questions/49987261/what-is-the-difference-between-cudnnlstm-and-lstm-in-keras">this post</a> that <code>CuDNNGRU</code> layers train faster than <code>GRU</code> layers, but 
  
  <ul>
  <li>Do the 2 layers converge to different results with the same seed? </li>
  <li>Do the 2 layers perform the same during inference? </li>
  <li>Do CuDNN layers require a GPU during inference?</li>
  <li>Can GRU layers run inference on a GPU?</li>
  <li>Are CuDNN layers easily deployable? I am currently using <code>coremlconverter</code> to convert my keras model to CoreML for deployment.</li>
  </ul></li>
  <li>Is there an equivalent CuDNN layer for <code>tf.keras.layers.SimpleRNN</code> (i.e. <code>tf.keras.layers.CuDNNSimpleRNN</code>)? I am not committed to a specific architecture yet, and so I believe I would need the <code>tf.keras.layers.CuDNNSimpleRNN</code> layer if I decide on SimpleRNNs and the CuDNN layer has some functionality that I need.</li>
  <li>With <code>CuDNN</code> layers, do I need to have <code>tensorflow-gpu</code> installed? Or do they still get deployed to the GPU as long as I have the relevant drivers installed?</li>
  </ol>