

Once the freeze notification is received, AutoPipe will always attempt to divide the pipeline length  by 2 (e.g., from 8 to 4, then 2). By using  as the input, the compression algorithm can verify if the result satisfies the criterion in Equation (1). Pseudocode is shown in lines 25-33 in Algorithm 1. Note that this compression makes the acceleration ratio exponentially increase during training, meaning that if a GPU server has a larger number of GPUs (e.g., more than 8), the acceleration ratio will be further amplified.


