1. In a computing environment, a system, comprising, a data deduplication pipeline including a chunking phase configured to split data of files into chunks, in which the chunking phase comprises one or more modules that each correspond to a chunking algorithm, a deduplication detection phase configured to determine for each chunk whether that chunk is already stored in a deduplication system, and a commit phase that commits chunks to the deduplication system that are not determined by the deduplication detection phase to be stored in the in the deduplication system, and commits reference data for chunks that are already determined to be stored in the deduplication system.