If a single epoch of the prepared dataset can't fit in memory, the BlockDataloader can provide efficient mitigation. The BlockDataloader loads data in chunks or blocks into memory from e.g. an HDF file that is too large for the machine memory. The chunck size can be chosen to fit the resource restraints, i.e. the amount of memory. The minibatches are then sampled randomly from the current in-memory Page 6 of 8 chunk. The dataloader discards the current block from memory and loads the next chunk when needed. If the chunks are sufficiently large, the random minibatches will probably be different between each epoch, while at the same time loading from disk into memory is efficient as the read is consecutive. The approach breaks with a full stochastic sampling of the dataset, but in practice we have not observed a difference in training and test performance if the dataset is pre-shuffled and the blocks are kept sufficiently large, as was done with the pre-processed graph tensors and action tensors in GraphInvent .