

We summarize the overall experimental results in the table above. Note that the speedup we report is based on a conservative   value that can obtain comparable or even higher accuracy. A more aggressive  (, ) can obtain a higher speedup but may lead to a slight loss in accuracy. Note that the model size of BERT (24 layers) is larger than ViT-B/16 (12 layers), thus it takes more time for communication.
Performance Analysis
Speedup Breakdown
This section presents evaluation results and analyzes the performance of different components in \autopipe. More experimental results can be found in the Appendix.
