# Pipeline Parallellism with Controllable Memory | |
Pipeline Parallelism with Controllable Memory creates a framework on designing pipeline schedules and uses the framework to find memory optimal efficient schedules. | |
From our findings, we need approximately 1/3 memory under ideal conditions (F, B and W have same runtime), and 1/2 memory to create zero bubble schedule in realistic scenarios (with the necessary condition being W + 2B ≥ 2F and W + 2F ≥ 2B ). | |
Check out our paper at [Arxiv](https://arxiv.org/abs/2405.15362). | |
| Comparison assuming T_F=T_B=T_W | 1F1B | V-Min | V-Half | V-ZB | | |
| ----------------------------------------------------- |-------|------- | ---------- | ---- | | |
| Bubble Rate | ~ p/m | ~ 2p/3m | ~ p/ 2m | 0 | | |
| Activation Memory <br> (Compared to 1F1B) | p | (p+4)/3 | (p+2)/2 | p | | |
Bubble Rate here is calculated as `1 - (F+B+W)*m / longest_stage_time`. |