title: Why is Keras LSTM on CPU three times faster than GPU?
body: <p>I use <a href="https://www.kaggle.com/sbongo/for-beginners-tackling-toxic-using-keras" rel="noreferrer">this notebook from Kaggle</a> to run LSTM neural network.</p>
  
  <p>I had started training of neural network and I saw that it is too slow. It is almost three times slower than CPU training.</p>
  
  <ul>
  <li><code>CPU perfomance:</code> 8 min per epoch;</li>
  <li><code>GPU perfomance:</code> 26 min per epoch.</li>
  </ul>
  
  <p>After this I decided to find answer in <a href="https://stackoverflow.com/questions/41972015/keras-lstm-on-cpu-faster-than-gpu">this question on Stackoverflow</a> and I applied a <code>CuDNNLSTM</code> <em>(which runs only on GPU)</em> instead of <code>LSTM</code>. </p>
  
  <p>Hence, GPU perfomance became <strong>only 1 min per epoch</strong> and accuracy of model decreased on 3%.</p>
  
  <p><strong>Questions:</strong> </p>
  
  <p>1) Does somebody know why GPU works slower than CPU in the classic <code>LSTM</code> layer? I do not understand why this happens.</p>
  
  <p>2) Why when I use <code>CuDNNLSTM</code> instead of <code>LSTM</code>, training become much more faster and the accuracy of the model decrease?</p>
  
  <p>P.S.:</p>
  
  <p><code>My CPU:</code> Intel Core i7-7700 Processor (8M Cache, up to 4.20 GHz)</p>
  
  <p><code>My GPU:</code> nVidia GeForce GTX 1050 Ti (4 GB)</p>