Comparing the performance of different task-specific LLMs presented in Figure , it can be seen that the MatSciBERT-based model consistently, on both test and validation sets, outperforms the SciBERT-based model which in turn performs better than the BERT model as well as the DeBERTa models, for which only the last layer was fine-tuned. This DeBERTa model's performance drops by approximately 20% when compared to the best-performing MatSciBERT model.