title: Are there any guidelines on sharding a data set?
body: <p>Are there any guidelines on choosing the number of shard files for a data set, or the number of records in each shard?</p>
  <p>In the examples of using <code>tensorflow.contrib.slim</code>,</p>
  <ul>
  <li><p>there are roughly <code>1024</code> records in each shard of <em>ImageNet</em> data set.(<a href="https://github.com/tensorflow/models/tree/master/inception" rel="nofollow noreferrer">tensorflow/models/inception</a>)</p>
  </li>
  <li><p>there are roughly <code>600</code> records in each shard of <em>flowers</em> data set. (<a href="https://github.com/tensorflow/models/blob/master/slim/datasets/download_and_convert_flowers.py" rel="nofollow noreferrer">tensorflow/models/slim</a>)</p>
  </li>
  </ul>
  <p>Do the number of shard files and the number of records in each shard has any impact on the training and the performance of the trained model?</p>
  <p>To my knowledge, if we don't split the data set into multiple shards, it will be not quite random for shuffling data as the <code>capacity</code> of the <code>RandomShuffleQueue</code> may be less than the size of the data set.</p>
  <p>Are there any other advantages of using multiple shards?</p>
  <hr />
  <p>Update</p>
  <p><a href="https://www.tensorflow.org/how_tos/reading_data/" rel="nofollow noreferrer">The documentation</a> says</p>
  <blockquote>
  <p>If you have more reading threads than input files, to avoid the risk that you will have two threads reading the same example from the same file near each other.</p>
  </blockquote>
  <p>Why can't we use 50 threads to read from 5 files?</p>