Validation loss diverging away from the training loss
This kind of overfitting is typical when finetuning large LMs.
The usual approaches to &quot;avoid&quot; it are:

Early stopping: select the checkpoint with the best validation loss.
Random restarts: train multiple times from scratch, and select the model with the best validation performance.