Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
Large Language Models;Catastrophic Forgetting;Neural Network Pruning
Thank you for Reviewer 53VB's review.

#  Q3: Why not use the relative portion of the pruning metric alone? It’s a complicated scoring metric, please ablate it.

In the paper, we presented comparative experiments with the pruning method that uses only “absolute change magnitude” (refer to Tables 3 and 4 in the paper). Here, we supplement the results of ablation experiments that use only “relative change magnitude” (only RCM) for pruning. We conducted the experiments on Llama3-8B, with the pruning ratio consistent with that in the paper, which is 90%.


| Tasks| Methods         | ceval   | gsm8k  | mmlu   | humaneval | Avg.   | Performance |
|-----|---------|---------|--------|--------|-----------|--------|-------------|
|RTE | Pre-trained model | 0.4386  | 0.7922 | 0.6594 | 0.5914    | 0.6204 | 0.819       |
|RTE | Full SFT          | 0.2311  | 0.075  | 0.2554 | 0.0       | 0.1403 | 0.890       |
|RTE |only RCM          | 0.4391  | 0.7915 | 0.6620 | 0.6097    | 0.6255 | 0.823       |
|RTE |FAPM              | 0.4623  | 0.7915 | 0.6454 | 0.5975    | 0.6242 | 0.897       |
|WikiQA | Pre-trained model | 0.4386  | 0.7922 | 0.6594 | 0.5914    | 0.6204 | 0.913       |
|WikiQA |Full SFT          | 0.2547  | 0.0    | 0.2422 | 0.0       | 0.1242 | 0.966       |
|WikiQA |only RCM          | 0.4385  | 0.7892 | 0.6589 | 0.6097    | 0.6240 | 0.947       |
|WikiQA |FAPM              | 0.4749  | 0.7975 | 0.6563 | 0.5853    | 0.6285 | 0.964       |
|Winogrande | Pre-trained model | 0.4386  | 0.7922 | 0.6594 | 0.5914    | 0.6204 | 0.519       |
|Winogrande |Full SFT          | 0.2792  | 0.0606 | 0.3438 | 0.0       | 0.1709 | 0.820       |
|Winogrande |only RCM          | 0.4369  | 0.7899 | 0.6597 | 0.5975    | 0.6209 | 0.551       |
|Winogrande |FAPM              | 0.4829  | 0.7680 | 0.6472 | 0.5731    | 0.6178 | 0.824       |
|SQuAD | Pre-trained model | 0.4386  | 0.7922 | 0.6594 | 0.5914    | 0.6204 | 0.371       |
|SQuAD |Full SFT          | 0.2806  | 0.0212 | 0.3206 | 0.0       | 0.1556 | 0.646       |
|SQuAD |only RCM          | 0.4449  | 0.7915 | 0.6582 | 0.5975    | 0.6230 | 0.374       |
|SQuAD |FAPM              | 0.4738  | 0.7310 | 0.6455 | 0.5748    | 0.6063 | 0.637       |

From the table, it can be seen that using only the relative change magnitude cannot ensure the performance on downstream tasks; the model performance is comparable to that of the pre-trained model. The reason is that pruning using the “relative change magnitude” is designed solely to address catastrophic forgetting. Using only the relative change magnitude results in retaining parameter values that are relatively small in the vector during pruning, which cannot guarantee the performance on downstream tasks.
To Reviewer 53VB (Continue)
Thank you for Reviewer 53VB's review.


# Q1: This paper proposes a complicated pruning metrics that only shows marginal improvement over prior art.

1. We cannot agree with your comment of "a complicated pruning metrics." This is because our method is sufficiently simple. FAPM only requires pruning the incremental model after fine-tuning, without necessitating any changes to the model training process, and our pruning procedure is also simple enough. To visually demonstrate the simplicity of FAPM, we have provided the PyTorch implementation code for the FAPM pruning metrics below. The formula (1) in the paper can be implemented in just a few lines of code.

```
def FAPM(W_pre, W_tv, key):
    """
    Computes the FAPM for a specific parameter in a model.

    Parameters:
    -----------
    W_pre : OrderedDict
        The model state_dict of the pre-trained model.
    
    W_tv : OrderedDict
        The model state_dict of the task vectors,
    
    key : str
        The identifier to specify which parameter matrix to use from both `W_pre` and `W_tv`.

    Returns:
    --------
    score : torch.Tensor
        The importance score (FAPM) for the specified parameter. 
    """

    # Compute the absolute value of the task vector for the specified parameter
    abs_W_tv = W_tv[key].abs()

    # Compute the mean of the absolute values of the pre-trained weights for the specified parameter
    mean_abs_W_pre = W_pre[key].abs().mean()

    # Compute the normalized absolute task vector
    normalized_abs_W_tv = abs_W_tv / W_pre[key].abs()

    # Compute the importance score (FAPM)
    score = abs_W_tv - mean_abs_W_pre * normalized_abs_W_tv

    return score
```

2. The improvements brought by our method are significant. FAPM effectively alleviates forgetting to a mere 1% while maintaining an impressive 99% accuracy on downstream tasks. Most importantly, compared to previous state-of-the-art methods, our greatest advantage lies in the simplicity of our approach. It does not require any changes to the model training process, does not necessitate adding extra model structures, and does not introduce additional data. Moreover, our method achieves results that are better than or comparable to the previous state-of-the-art methods (refer to Table 1 and Table 2 in the paper).

3. The simplicity and effectiveness of our method have been fully recognized by other reviewers, e.g., *"The simplicity of the technique introduced by the authors is really great"* commented  by Reviewer YFJm and *"FAPM doesn’t complicate the model architecture or training process, making it easier to implement"* commented by Reviewer HmBx. Therefore, we believe that Reviewer 53VB's statement regarding our method as "a complicated pruning metric that only shows marginal improvement over prior art" is not appropriate. We cannot agree with this assessment.

# Q2: The quality of write-up is generally poor. For example, section 3.1 is entirely unnecessary and can be in the appendix without affecting the flow of the paper.

We do not believe that Section 3.1 is unnecessary because this section presents our analysis of the causes of the CF problem, elucidates our perspective on the issue, and introduces the starting point of our method. We believe these analyses are necessary and insightful. As mentioned by Reviewer YFJm, *"I loved reading section 3.1 wherein they provide analysis and a clear thinking process of how they arrived at the method"*, this section helps readers to better understand our motivation.
To Reviewer 53VB
Yes
No ethics review needed.
1. Nowadays most people prefer to use LoRA not because it reduces CF, but instead its pretty cheap. In my experience even LoRA suffers from CF if the finetuning domain is pretty niche. So I was wondering how informative are task vectors from LoRA merged models and how will this method perform if we use LoRA to finetune.
1. The simplicity of the technique introduced by the authors is really great. Also I loved reading section 3.1 wherein they provide analysis and a clear thinking process of how they arrived at the method.

2. There are extensive experiments in the paper across various tasks and two models. The models chose are the latest ones which makes this paper a bit more relevant. Section 5 in general is pretty enjoyable to read.

3. Although the method is not pareto optimal but the numbers are consistently high on all the tasks across both Llama3 and Qwen2.
This paper tries to tackle the problem of whether catastrophic forgetting (CF) due to finetuning be avoided without changing training process, without any additional data, and without altering model structure. To this end the authors come up with Forgetting-Aware Pruning Metric (FAPM) wherein the task vectors are not only pruned based on magnitude but also uses the ratio of task vector to pretrained parameters to avoid CF. The authors provide extensive experiments across various tasks and latest LLMs like Qwen and Llama3 models.
1. A small nitpick. It would be really great if the captions of the images and tables could be a bit longer and more informative.
Yes
No ethics review needed.
1. This appears to be a unique model merging method; however, why is there no comparison made with other methods of a similar nature?
2. While you have elucidated how to control FAPM to achieve a better balance between accuracy and forgetting levels, is there a more effective approach to directly estimate an optimal balance point rather than adjusting parameters through multiple rounds of performance feedback?
1. FAPM effectively limits forgetting to a mere 1%, which is a significant improvement.
2. The method maintains an impressive 99% accuracy in downstream tasks, showing that it preserves performance.
3. Unlike some existing methods, FAPM doesn’t complicate the model architecture or training process, making it easier to implement.
4. This approach doesn’t rely on replaying original training data or using additional datasets, making it more practical in various scenarios.
In this paper, the authors introduce Forgetting-Aware Pruning Metric (FAPM), a new pruning-based method aimed at striking a balance between reducing forgetting and maintaining downstream task performance. They find that the overlap between task vectors – which represent the difference between pre-trained weights and those fine-tuned on new tasks – and pre-trained model parameters is a major factor in forgetting. FAPM uses this overlap ratio as a metric for assessing forgetting and incorporates it into pruning decisions. A key advantage is that FAPM doesn't require changes to the training process or the model architecture, nor does it need any additional data. The authors conducted experiments across six datasets covering various tasks, and the results show that FAPM keeps forgetting to just 1% while achieving 99% accuracy on downstream tasks, making it very competitive against other state-of-the-art methods.
1. The reliance on the overlap between task vectors and pre-trained weights might not generalize across all types of tasks or models.
2. While pruning is effective, it may not address all aspects of learning or forgetting, so there could be other methods worth exploring along with FAPM.
3. While the experiments are comprehensive, testing on even more diverse datasets could strengthen the generalizability of the findings.
4. The paper doesn’t discuss how FAPM performs in highly specialized or unusual task scenarios, which might reveal limitations.
Yes
No ethics review needed.
- See weakness.
- Using pruning to mitigate catastrophic forgetting during fine-tuning is novel.
- This paper contains some original insights about the generality/specificity trade-off when fine-tuning LLMs.
The authors identified pruning as a way to mitigate catastrophic forgetting. Specifically, the authors propose to prune weight updates due to fine-tuning, using a combination of pruning metrics including absolute magnitude of change in weights and relative magnitude of change in weights relative to the magnitude of pre-trained weights.
- This paper proposes a complicated pruning metrics that only shows marginal improvement over prior art.
- The quality of write-up is generally poor. For example, section 3.1 is entirely unnecessary and can be in the appendix without affecting the flow of the paper.
-  No ablation study. Why not use the relative portion of the pruning metric alone? It’s a complicated scoring metric, please ablate it.
Yes
No ethics review needed.
1.The limitation mentioned in lines 81-83, stating that "methods that alter the training process or model architecture not only make the training process more difficult to control but also degrade the accuracy of downstream tasks," lacks references to prior work supporting this claim.

2.The figures in this paper appear not to be in vector graphic format, which may affect the clarity and quality of the visualizations.

3.The quotation marks in line 212 are not properly formatted according to standard usage.
1.This paper introduces two key concepts through preliminary experiments—“absolute change magnitude” and “relative change magnitude”—which are designed to balance the trade-off between task specialization and generality.

2.Building on these concepts, the paper proposes the Forgetting-Aware Pruning Metric (FAPM), aimed at deriving an optimal task vector to obtain the final model weights.

3.Extensive experiments demonstrate the effectiveness of FAPM, showing significant improvements over baseline methods.
This paper focuses on addressing the issue of catastrophic forgetting (CF) in large language models (LLMs) during fine-tuning, i.e., striking a balance between speciality and generality.
It proposes Forgetting-Aware Pruning Metric (FAPM): A novel method that integrates the ratio of task vector magnitude to pre-trained model parameters into the pruning criteria, allowing for the mitigation of CF without modifying the model architecture or training process.
FAPM successfully mitigates forgetting while retaining 99% of downstream task accuracy across various natural language tasks.
The experiments show that FAPM performs competitively against state-of-the-art methods.
1.This paper overlooks an important baseline—Wise-FT [1], a standard model parameter merging method.

2.The experimental setup for the preliminary experiments, particularly those in Sections 2 and 3, is not clearly detailed, e.g., the experimental model, raising concerns about whether similar observations would hold for other models.

3.While instruction-following is a critical aspect of generality, this paper focuses solely on world knowledge and generic reasoning, omitting this aspect.

4.The rationale and motivation for striking a balance between speciality and generality is not clearly articulated in the paper.

[1] Mitchell Wortsman, Gabriel Ilharco, Mike Li, Jong Wook Kim, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. 2021. Robust fine-tuning of zero-shot models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7949–7961.
We propose Forgetting-Aware Pruning Metric, a novel pruning-based approach to balance Catastrophic Forgetting and downstream task performance for LLMs.
@inproceedings{
anonymous2024mitigating,
title={Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=fHvh913U1H},
note={under review}
}
Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. These models are typically pretrained on extensive corpora and subsequently fine-tuned on task-specific datasets. However, during the fine-tuning process, LLMs often suffer from Catastrophic Forgetting (CF), wherein previously acquired general knowledge is lost. Traditional approaches to mitigating CF often rely on data replay, which may not be viable when the original training data is inaccessible. Additionally, methods that alter the training process or the model architecture can increase complexity and detract from the accuracy of downstream tasks, thus limiting their generalizability. In this paper, we propose Forgetting-Aware Pruning Metric (FAPM), a novel pruning-based approach to balance CF and downstream task performance. Our investigation reveals that the degree to which task vectors (i.e., the subtraction of pre-trained weights from the weights fine-tuned on downstream tasks) overlap with pre-trained model parameters is a critical factor for CF. Motivated by this insight, FAPM employs the ratio of the task vector to pre-trained model parameters as a metric to quantify CF, integrating this measure into the pruning criteria. Importantly, FAPM does not necessitate modifications to the training process or model architecture, nor does it require any auxiliary data. We conducted extensive experiments across six datasets encompassing natural language inference, question answering, reading comprehension, and cloze tests. The results demonstrate that FAPM limits CF to just 1% while maintaining 99% accuracy on downstream tasks, rendering FAPM highly competitive relative to the state-of-the-art methods that involve modifications to the training process.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Models Catastrophic Forgetting Neural Network Pruning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/a8dcb2c6c82691cf7ed55afa8c7fa4c6b1ffefdf.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission