MLPerf v1.0 training workloads
The PyTorch CUDA graphs functionality was instrumental in scaling NVIDIA’s MLPerf training v1.0 workloads (implemented in PyTorch) to over 4000 GPUs, setting new records across the board. We illustrate below two MLPerf workloads where the most significant gains were observed with the use of CUDA graphs, yielding up to ~1.7x speedup.




Number of GPUs
Speedup from CUDA-graphs




Mask R-CNN
272
1.70×


BERT
4096
1.12×



Table 1. MLPerf training v1.0 performance improvement with PyTorch CUDA graph.
Mask R-CNN