| We showcase the building block of each alternative schedules below. | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/646968e05d7015663950e95b/TXf74JGHaqanCcMm_jPbw.jpeg" height="100px"/> | |
| <center>1F1B-V</center> | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/646968e05d7015663950e95b/TCCgzQEwjLDO3_12OUzmN.jpeg" height="100px"/> | |
| <center>Zero Bubble schedule with 2/3 1F1B memory</center> | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/646968e05d7015663950e95b/AwXeE4e0HO20u_YJV23AN.jpeg" height="100px"/> | |
| <center>Variation of Interleaved 1F1B schedule</center> | |