title: Tensorflow: Dataset Map With num_parallel_calls Offers No Speedup
body: <p>I'm using TensorFlow and the <code>tf.data.Dataset</code> API to perform some text preprocessing.  Without using <code>num_parallel_calls</code> in my <code>dataset.map</code> call, it takes 0.03s to preprocess 10K records.</p>
  
  <p>When I use <code>num_parallel_trials=8</code> (the number of cores on my machine), it also takes 0.03s to preprocess 10K records.</p>
  
  <p>I googled around and came across this: <a href="https://stackoverflow.com/questions/48484689/parallelism-isnt-reducing-the-time-in-dataset-map">Parallelism isn&#39;t reducing the time in dataset map</a></p>
  
  <p>where they indicate that you need to use TensorFlow operations to see a speedup.  Here's the thing: I <em>am</em> using only TensorFlow operations.  Specifically, I'm mapping this function:</p>
  
  <pre><code>def preprocess(self, x, data_table):
      x['reviews'] = tf.string_split(x['reviews'], delimiter=' ')
      x['reviews'] = tf.sparse_tensor_to_dense(x['reviews'], default_value=' ')
      x['reviews'] = tf.cast(data_table.lookup(x['reviews']), tf.int32)
      nbatch = tf.cast(tf.shape(x['reviews'])[0], tf.int32)
      nseq = tf.cast(tf.shape(x['reviews'])[1], tf.int32)
      padding = tf.cond(tf.less(nseq, 100),
                        lambda: 0 * tf.ones([nbatch, 100 - nseq], tf.int32),
                        lambda: 0 * tf.ones([nbatch, 0], tf.int32))
      x['reviews'] = tf.concat((x['reviews'], padding), axis=1)[:, :100]
      x['reviews'].set_shape([None, 100])
      return x
  </code></pre>
  
  <p>Any idea why I don't see any speedup?</p>
  
  <p>Thanks!</p>