Need to kickstart learning rates
This technique of gradually increasing the learning rate linearly during early phase of training is called learning rate warmup.
This is typically needed in order to prevent overfitting during the initial phase of training where a certain subset of observations may significantly skew your model towards some features and push you towards a bad local optimum.
The rectified adam paper suggests that this warmup heuristic serves as a variance reduction technique.

Due to the lack of samples in the early stage, the adaptive learning rate has an undesirably large variance, which leads to suspicious/bad local optima

They perform a few experiments that validates their hypothesis.