Pre-trained language models based on the transformer architecture achieve state-of-the-art performance when fine-tuned on token-classification tasks such as NER. Language models are typically pre-trained using self-supervised learning on large unlabelled text corpora. During pre-training, they capture patterns connecting words in the training corpus. When these models are used for a NER downstream task, they are extended with a classification head and the model is fine-tuned using an annotated NER dataset. A few widely-used fine-tuned language models for NER tasks include BERT , RoBERTa , and DeBERTa . For different domains, specialized pre-trained states of BERT are developed that differ mainly in the pre-training corpus, such as SciBERT trained on scientific publications with a focus on the biomedical domain, and MatSciBERT 14 trained on materials science research articles. The fine-tuning of language models can be computationally expensive depending on the model size. This motivated growing research on alternate fine-tuning strategies that could reduce trainable parameters of the model, collectively called parameter efficient fine-tuning (PEFT) . One prominent PEFT technique is low-rank adaptation (LoRA) , which approximates weight updates of the language model into matrices of low rank. Few PEFT methods, including LoRA, are shown to perform better than full fine-tuning in low-data scenarios , which makes it desirable for tasks in materials science, where annotated data is expected to be sparse.