Overcoming Catastrophic Forgetting: A Novel Fine-Tuning Method
lifelong learning
This paper is desk rejected for significant textual overlap with [1]. The introduction has significant overlap in text, and method section has significant overlap in text, both of which are without proper citation. This level of overlap is considered plagiarism. The decision was confirmed by multiple members of the program committee. 

[1] "Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models" by Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang. https://arxiv.org/pdf/2311.08011
Submission Desk Rejected by Program Chairs
Updating knowledge by updating the most relevant parameters.
@misc{
ding2024overcoming,
title={Overcoming Catastrophic Forgetting: A Novel Fine-Tuning Method},
author={Fei Ding},
year={2024},
url={https://openreview.net/forum?id=F8BPhZ5nmU}
}
Despite remarkable advances in Large Language Models (LLMs), a persistent challenge remains: the potential for these models to acquire erroneous or outdated information from their training data. Direct fine-tuning with data containing new knowledge can be ineffective due to conflicts between old and new knowledge. This paper proposes a novel fine-tuning paradigm called Delicate Fine-Tuning (DFT ) that leverages parametric arithmetic to pinpoint the location of knowledge and update only the minimal set of relevant parameters. Experimental results on two publicly available datasets demonstrate that our proposed DFT significantly improves the knowledge updating performance of full fine-tuning, consistently outperforming existing baselines in most cases.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Fei_Ding6
Fei Ding
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
lifelong learning
I certify that there is no acknowledgement section in this submission for double blind review.
ding|overcoming_catastrophic_forgetting_a_novel_finetuning_method
/pdf/c255593459a7bd978266d7b1868789870dc52535.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Overcoming Catastrophic Forgetting: A Novel Fine-Tuning Method
ICLR 2025 Conference Desk Rejected Submission
ICLR.cc/2025/Conference/Desk_Rejected_Submission