
Figure 10. Tuning  in Freezing Algorithm

We ran experiments to show how the   in the freeze algorithms influences training speed. The result clearly demonstrates that a larger  (excessive freeze) leads to a greater speedup but suffers from a slight performance degradation. In the case shown in Figure 10, where , freeze training outperforms normal training and obtains a -fold speedup. We provide more results in the Appendix.
Optimal Chunks in the elastic pipeline



Figure 11. Optimal chunk number in the elastic pipeline
