title: How to read data into TensorFlow batches from example queue?
body: <p>How do I get TensorFlow example queues into proper batches for training?</p>
  
  <p>I've got some images and labels:</p>
  
  <pre><code>IMG_6642.JPG 1
  IMG_6643.JPG 2
  </code></pre>
  
  <p>(feel free to suggest another label format; I think I may need another dense to sparse step...)</p>
  
  <p>I've read through quite a few tutorials but don't quite have it all together yet.
  Here's what I have, with comments indicating the steps required from TensorFlow's <a href="https://www.tensorflow.org/versions/r0.8/how_tos/reading_data/index.html#reading-from-files" rel="noreferrer">Reading Data</a> page.</p>
  
  <ol>
  <li>The list of filenames
  (optional steps removed for the sake of simplicity)</li>
  <li>Filename queue</li>
  <li>A Reader for the file format</li>
  <li>A decoder for a record read by the reader</li>
  <li>Example queue</li>
  </ol>
  
  <p>And after the example queue I need to get this queue into batches for training; that's where I'm stuck...</p>
  
  <p><strong>1. List of filenames</strong></p>
  
  <p><code>files = tf.train.match_filenames_once('*.JPG')</code></p>
  
  <p><strong>4. Filename queue</strong></p>
  
  <p><code>filename_queue = tf.train.string_input_producer(files, num_epochs=None, shuffle=True, seed=None, shared_name=None, name=None)</code></p>
  
  <p><strong>5. A reader</strong></p>
  
  <p><code>reader = tf.TextLineReader()
  key, value = reader.read(filename_queue)</code></p>
  
  <p><strong>6. A decoder</strong></p>
  
  <p><code>record_defaults = [[""], [1]]
  col1, col2 = tf.decode_csv(value, record_defaults=record_defaults)
  </code>
  (I don't think I need this step below because I already have my label in a tensor but I include it anyways)</p>
  
  <p><code>features = tf.pack([col2])</code></p>
  
  <p>The documentation page has an example to run one image, not get the images and labels into batches:</p>
  
  <p><code>
  for i in range(1200):
      # Retrieve a single instance:
      example, label = sess.run([features, col5])
  </code></p>
  
  <p>And then below it has a batching section:</p>
  
  <pre><code>def read_my_file_format(filename_queue):
    reader = tf.SomeReader()
    key, record_string = reader.read(filename_queue)
    example, label = tf.some_decoder(record_string)
    processed_example = some_processing(example)
    return processed_example, label
  
  def input_pipeline(filenames, batch_size, num_epochs=None):
    filename_queue = tf.train.string_input_producer(
    filenames, num_epochs=num_epochs, shuffle=True)
    example, label = read_my_file_format(filename_queue)
    # min_after_dequeue defines how big a buffer we will randomly sample
    #   from -- bigger means better shuffling but slower start up and more
    #   memory used.
    # capacity must be larger than min_after_dequeue and the amount larger
    #   determines the maximum we will prefetch.  Recommendation:
    #   min_after_dequeue + (num_threads + a small safety margin) *              batch_size
    min_after_dequeue = 10000
    capacity = min_after_dequeue + 3 * batch_size
    example_batch, label_batch = tf.train.shuffle_batch(
    [example, label], batch_size=batch_size, capacity=capacity,
    min_after_dequeue=min_after_dequeue)
    return example_batch, label_batch
  </code></pre>
  
  <p>My question is: <strong>how do I use the above example code with the code I have above?</strong>  I need <em>batches</em> to work with, and most of the tutorials come with mnist batches already.</p>
  
  <pre><code>with tf.Session() as sess:
    sess.run(init)
  
    # Training cycle
  for epoch in range(training_epochs):
      total_batch = int(mnist.train.num_examples/batch_size)
      # Loop over all batches
      for i in range(total_batch):
          batch_xs, batch_ys = mnist.train.next_batch(batch_size)
  </code></pre>