How to combine multiple h5 files?
Limited by the device, I could only produce several h5 files (the format of each file are same with shape of [idx, 1, 224, 224]) for huge dataset (>100GB) and now I'm confused about the solution to combine these files into a single one for further training on PyTorch. enter image description here

In h5py, groups and files support copy(), which can be used to move groups (including the root group) and their contents between files.

See the docs here (scroll down a bit to find copy()):

http://docs.h5py.org/en/latest/high/group.html

The HDF5 distribution also includes a command-line tool called h5copy that can be used to move things around, and the C API has an H5Ocopy() function.
