| Local Cluster with existing distributed FS or object store |  WebDataset’s large sequential reads improve performance with existing distributed stores and eliminate the need for dedicated volume plugins. |
| Educational Environments | WebDatasets can be stored on existing web servers and web caches, and can be accessed directly by students by URL |
| Training on Workstations from Local Drives | Jobs can start training as the data still downloads. Data doesn’t need to be unpacked for training. Ten-fold improvements in I/O performance on hard drives over random access file-based datasets. |
| All Environments | Datasets are represented in an archival format and contain metadata such as file types. Data is compressed in native formats (JPEG, MP4, etc.). Data management, ETL-style jobs, and data transformations and I/O are simplified and easily parallelized.  |
We will be adding more examples giving benchmarks and showing how to use WebDataset in these environments over the coming months.
High-Performance