title: Prediction with tensorflow-gpu is slower than tensorflow-cpu
body: <p>Below is my code for prediction:</p>
  
  <pre><code>start=time.time()
  with tf.Session(graph=graph) as sess:
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
      predict('/home/4_bikes/test_images/bikerider4.jpg',sess)    
  stop=time.time()
  print('Time taken for prediction :: {}'.format(stop-start))
  </code></pre>
  
  <p>Below is my <code>predict</code> function :</p>
  
  <pre><code>def predict(file_name,sess):
  
    t = read_tensor_from_image_file(
        file_name,
        input_height=input_height,
        input_width=input_width,
        input_mean=input_mean,
        input_std=input_std)
  
    results = sess.run(output_operation.outputs[0], {
          input_operation.outputs[0]: t
      })
    results = np.squeeze(results)
  
    index=results.argmax()
  
    prediction=labels[index]
    bike_predictor = bike_classifier()
    if prediction == 'bikes':
      bike_predictor.predict(t)
    else:
      print('Predicted as :: unknown')
  </code></pre>
  
  <p>I have installed tensorflow-gpu on python-2 and tensorflow-cpu on python-3. When I run it with tensorflow-gpu I get:</p>
  
  <pre><code>Time taken for prediction :: 2.92091107368
  </code></pre>
  
  <p>When I run with tensorflow-cpu I get:</p>
  
  <pre><code>Time taken for prediction :: 1.7942276000976562
  </code></pre>
  
  <p>I am sure I am using GPU because when running with python-2 I get log:</p>
  
  <pre><code>name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6705
  pciBusID: 0000:01:00.0
  totalMemory: 10.91GiB freeMemory: 10.28GiB
  2018-05-31 18:23:26.762628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
  2018-05-31 18:23:26.906629: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
  2018-05-31 18:23:26.906672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
  2018-05-31 18:23:26.906679: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
  2018-05-31 18:23:26.906856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9949 MB memory) -&gt; physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
  </code></pre>
  
  <p>each time <code>predict</code> is called.</p>
  
  <p>Can someone help me with this? Where am I wrong? when using GPU time taken should be less.</p>
  
  <p>For installation I followed <a href="https://github.com/markjay4k/Install-Tensorflow-on-Ubuntu-17.10-/blob/master/Tensorflow%20Install%20instructions.ipynb" rel="nofollow noreferrer">this</a> link.</p>
  
  <p>I am using <strong>Nvidia GeForce GTX 1080 Ti</strong>. </p>
  
  <p>CPU is <strong>Intel(R) Core(TM) i7-7700K CPU</strong></p>
  
  <p>Model is <strong>MobileNet_v1</strong></p>