title: GPU under utilization using tensorflow dataset
body: <p><a href="https://i.stack.imgur.com/oyDNI.png" rel="noreferrer"><img src="https://i.stack.imgur.com/oyDNI.png" alt="TF Trace"></a></p>
  
  <p>During training of my data, my GPU utilization is around 40%, and I clearly see that there is a datacopy operation that's using a lot of time, based on tensorflow profiler(see attached picture). I presume that "MEMCPYHtoD" option is copying the batch from CPU to GPU, and is blocking the GPU from being used. Is there anyway to prefetch data to GPU? or is there other problems that I am not seeing?</p>
  
  <p>Here is the code for dataset:</p>
  
  <pre><code>X_placeholder = tf.placeholder(tf.float32, data.train.X.shape)
  y_placeholder = tf.placeholder(tf.float32, data.train.y[label].shape)
  
  dataset = tf.data.Dataset.from_tensor_slices({"X": X_placeholder, 
                                                "y": y_placeholder})
  dataset = dataset.repeat(1000)
  dataset = dataset.batch(1000)
  dataset = dataset.prefetch(2)
  iterator = dataset.make_initializable_iterator()
  next_element = iterator.get_next()
  </code></pre>