Two different fine-tuning strategies were tested: Low-Rank Adaptation (LoRA) as Parameter-Efficient Fine-Tuning (PEFT) method and regular fine-tuning. Briefly, instead of directly updating weight matrices W or using an additional ∆W accumulating gradient update as done in regular fine-tuning, LoRA uses two smaller weight matrices A and B approximating ∆W when multiplied, making fine-tuning computationally less demanding. The dimension of the weight matrices A and B is determined by the rank r, a hyperparameter typically set to 4, 8, or 16. Following guidelines from the original LoRA publication, we applied LoRA only to the query W q and value W v weight matrices in each self-attention layer of the pretrained pLMs, setting r to either 4, 8 or 16. As well as LoRA, only the query W q and value W v weight matrices were updated for the regular fine-tuning procedure, but only in the last self-attention layer, while keeping all the other parameters fixed.