title: How to determine the optimal configuration for batch inputs in tensorflow?
body: <p>Please first see the following code snippets: </p>
  
  <pre><code>def inputs(filenames, batch_size, num_epochs, shuffle=True):
  
  with tf.name_scope('input'):
      filename_queue = tf.train.string_input_producer(filenames, 
                                                      shuffle=shuffle,
                                                      capacity=32*batch_size,
                                                      num_epochs=num_epochs)
  
  
      if shuffle:
          image, label = read_and_decode_with_random_processing(filename_queue)
          images, sparse_labels = tf.train.shuffle_batch([image, label], 
                                                         batch_size=batch_size, 
                                                         num_threads=32,
                                                         capacity= 16 * batch_size,
                                                         min_after_dequeue= 8 * batch_size)
      else:
          image, label = read_and_decode(filename_queue)
          images, sparse_labels = tf.train.batch([image, label], 
                                                 batch_size=batch_size, 
                                                 num_threads=32,
                                                 capacity=8 * batch_size,
                                                 allow_smaller_final_batch=True)
  
      return images, sparse_labels
  </code></pre>
  
  <p>there are three <code>capacity</code> variables and two <code>num_threads</code> variables. In my practice, I found these guys are hard to determine to let the GPUs be busy-working. It seems the GPU works too faster than these guys to load the data. How do you determine these numbers in practice? In my opinion, the larger these numbers are, the better it should be (because the queue is always full of data, the GPUs don't need to wait). But if I set these numbers too big, it is slow for the prefetch. Maybe I missed something. </p>