

tarp is a small command-line program for splitting, merging, shuffling, and processing tar archives and WebDataset datasets.


tensorcom is a library supporting distributed data augmentation and RDMA to GPU.


pytorch-imagenet-wds contains an example of how to use WebDataset with ImageNet, based on the PyTorch ImageNet example.


Bigdata 2019 Paper with Benchmarks


Check out the library and provide your feedback for RFC 38419.