<p dir="auto">it works with make_one_shot_iterator(), I can see that the training time is 2x lower, when using the make_initializable_iterator() the time is the same as without prefetching to device.<br>
how to use the new feature when dataset switch (train/val/test) is needed for evaluation purpose?<br>
code parts<br>
//not working<br>
trainDataset = tf.data.Dataset.from_tensor_slices((trainFeatures,trainLabels,trainLengths,trainMasks))<br>
trainDataset = trainDataset.batch(batchSize)<br>
trainDataset = trainDataset.apply(tf.contrib.data.shuffle_and_repeat(100,nEpochs))<br>
iterator = tf.data.Iterator.from_structure(trainDataset.output_types, trainDataset.output_shapes)<br>
train_init_op = iterator.make_initializer(trainDataset)<br>
trainDataset = trainDataset.apply(tf.contrib.data.prefetch_to_device('/gpu:0'))</p>
<p dir="auto">//working<br>
trainDataset = tf.data.Dataset.from_tensor_slices((trainFeatures,trainLabels,trainLengths,trainMasks))<br>
trainDataset = trainDataset.batch(batchSize)<br>
trainDataset = trainDataset.apply(tf.contrib.data.shuffle_and_repeat(100,nEpochs))<br>
trainDataset = trainDataset.apply(tf.contrib.data.prefetch_to_device('/gpu:0'))<br>
iterator = trainDataset.make_one_shot_iterator()</p>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>:<br>
Yes</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: Linux Ubuntu 16.04</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: binary</li>
<li><strong>TensorFlow version (use command below)</strong>: 1.8.0</li>
<li><strong>Python version</strong>: 2.7.12</li>
<li><strong>Bazel version (if compiling from source)</strong>:</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>:</li>
<li><strong>CUDA/cuDNN version</strong>:</li>
<li><strong>GPU model and memory</strong>:</li>
<li><strong>Exact command to reproduce</strong>:</li>
</ul>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">I'm using <strong>tf.data.Iterator.from_structure</strong> mechanism to switch between training/validation datasets during training<br>
However, when <strong>tf.contrib.data.prefetch_to_device</strong> is added at the end of the dataset pipeline, the code crashes with the following error:<br>
<code class="notranslate">NotImplementedError: `prefetch_to_device()` must be the last transformation in a dataset pipeline.</code><br>
Explicit one shot iterator works fine</p>
<h3 dir="auto">Source code / logs</h3>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="with tf.Graph().as_default():
    with tf.Session() as sess:
        dataset = tf.data.Dataset.range(10).apply(tf.contrib.data.prefetch_to_device('/cpu:0'))

        # 1) works fine
        iterator = dataset.make_one_shot_iterator()
        print sess.run(iterator.get_next())
        
        # 2) fails
        iterator = tf.data.Iterator.from_structure((tf.int64), ([]))
        sess.run(iterator.make_initializer(dataset))
        print sess.run(iterator.get_next())"><pre class="notranslate"><code class="notranslate">with tf.Graph().as_default():
    with tf.Session() as sess:
        dataset = tf.data.Dataset.range(10).apply(tf.contrib.data.prefetch_to_device('/cpu:0'))

        # 1) works fine
        iterator = dataset.make_one_shot_iterator()
        print sess.run(iterator.get_next())
        
        # 2) fails
        iterator = tf.data.Iterator.from_structure((tf.int64), ([]))
        sess.run(iterator.make_initializer(dataset))
        print sess.run(iterator.get_next())
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="0

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
&lt;ipython-input-34-dbae58b82de0&gt; in &lt;module&gt;()
      6 
      7         iterator = tf.data.Iterator.from_structure((tf.int64), ([]))
----&gt; 8         sess.run(iterator.make_initializer(dataset))
      9         print sess.run(iterator.get_next())

/usr/local/lib/python2.7/dist-packages/tensorflow/python/data/ops/iterator_ops.pyc in make_initializer(self, dataset, name)
    306     with ops.colocate_with(self._iterator_resource):
    307       return gen_dataset_ops.make_iterator(
--&gt; 308           dataset._as_variant_tensor(), self._iterator_resource, name=name)  # pylint: disable=protected-access
    309 
    310   def get_next(self, name=None):

/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/data/python/ops/prefetching_ops.pyc in _as_variant_tensor(self)
    289     # TODO(mrry): Investigate support for chaining further transformations after
    290     # the prefetch, including GPU support.
--&gt; 291     raise NotImplementedError(&quot;`prefetch_to_device()` must be the last &quot;
    292                               &quot;transformation in a dataset pipeline.&quot;)
    293 

NotImplementedError: `prefetch_to_device()` must be the last transformation in a dataset pipeline."><pre class="notranslate"><code class="notranslate">0

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
&lt;ipython-input-34-dbae58b82de0&gt; in &lt;module&gt;()
      6 
      7         iterator = tf.data.Iterator.from_structure((tf.int64), ([]))
----&gt; 8         sess.run(iterator.make_initializer(dataset))
      9         print sess.run(iterator.get_next())

/usr/local/lib/python2.7/dist-packages/tensorflow/python/data/ops/iterator_ops.pyc in make_initializer(self, dataset, name)
    306     with ops.colocate_with(self._iterator_resource):
    307       return gen_dataset_ops.make_iterator(
--&gt; 308           dataset._as_variant_tensor(), self._iterator_resource, name=name)  # pylint: disable=protected-access
    309 
    310   def get_next(self, name=None):

/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/data/python/ops/prefetching_ops.pyc in _as_variant_tensor(self)
    289     # TODO(mrry): Investigate support for chaining further transformations after
    290     # the prefetch, including GPU support.
--&gt; 291     raise NotImplementedError("`prefetch_to_device()` must be the last "
    292                               "transformation in a dataset pipeline.")
    293 

NotImplementedError: `prefetch_to_device()` must be the last transformation in a dataset pipeline.
</code></pre></div>
<p dir="auto">I guess, prefetching is still under development? Is there a temporary workaround for this kind of scenario?</p>