Additionally, it can be observed from Figure that the performance of all models on the validation dataset (in-distribution performance) is much better than their performance on the test dataset (out-of-distribution performance). All models show an in-distribution to out-of-distribution performance decrease ranging between 14% and 20%. This indicates that all the models presented do not generalize well to out-of-distribution examples. Among fine-tuning strategies, better generalization performance was observed for LoRA fine-tuning compared to full fine-tuning. For instance, the full fine-tuned BERT exhibited a performance decline of 18% while the corresponding LoRA fine-tuning showed a decreased of only 15%. Additionally, the generalization performance of ICL was observed to be worse than that of task-specific models -with the in-context GPT-4 declining by 17% compared to the conventionally fine-tuned MatSciBERT showing a decline of 15%. A heatmap of the class-wise average F1-scores of each model for the test dataset is presented in Figure . The highest F1-score value for each entity type is additionally mentioned in the corresponding column. The rows are sorted by number of instances in the training dataset (numeric values in parentheses contained in the tick labels). With a few exceptions, the fully fine-tuned MatSciBERT-based NER model reaches the best performance for the majority of entity types, mostly in domain-specific entity types such as "Defect", "Damage", or "CrystallographicEntity". SciB-ERT claims the best F1-scores in most of the remaining types, followed by in-context GPT-4 and BERT. Overall, "Value", "Mechan-icalProperty", and "EnvironmentalTestParameter" types yielded the best results. Further, all models performed poorly on "Sha-peDescriptor" and "MicrostructuralProperty". For entity types which are supported with few training instances, task-specialized models fall sometimes short of ICL.