title: Storing lots of small files: archive vs. filesystem
body: <p>I am creating an application that requires a lot of image thumbnails (~3000, 5-25KB). Because speed is essential I plan on loading these images into memory when the application starts. At runtime, new thumbnails will be downloaded and added to the collective.</p>
  
  <p>I could store them all in a folder, but reading thousands of files into memory when a program starts hardly seems efficient.</p>
  
  <p>My second option would be to save them in some kind of (compressed) archive. This would make storage itself and loading more efficient (I think). However, new files will be added regularly, and that will probably not go as smoothly as just saving them in a folder.</p>
  
  <p>Is storing a cache of small files in a (compressed) archive a bad idea or not? Are ZIP files the way to go? Would I be better off using uncompressed archives (and if so, what kind)?</p>
  
  <p>All image files will be JPEG's.</p>
  
  <p>Thanks in advance!</p>
  
  <p>EDIT: I am considering to drop the "load everything into memory on application start" thing. This would simplify my question a little. My initial idea to put everything in one big file now seems less beneficial, since the problem of many files in one directory can be solved by hashing into subdirectories.</p>