Backdoor Attacks for LLMs with Weak-To-Strong Knowledge Distillation
Backdoor Attacks;Large Language Models;Knowledge Distillation
Yes
No ethics review needed.
1. The paper addresses injecting clean-label backdoors into LLMs under the assumption that the attacker has full control over the training process, making this setup counterintuitive and somewhat confusing. The main advantage of clean-label backdoor attacks is their stealthiness, as they can bypass human inspection. Most existing clean-label backdoor attacks operate under a data poisoning assumption, where the attacker only provides poisoned data without controlling the training process [1, 2, 3, 4]. In this scenario, model trainers may inspect the received data before using it for training. Due to the label consistency in clean-label backdoor attacks, simple human inspection cannot detect the poisoned samples, which makes them stealthy. However, in a training control setup [5, 6], the stealth advantage of a clean-label backdoor is irrelevant because the attacker will only release the poisoned model, without exposing the poisoned training data. This means there is no data inspector in such a scenario, and attackers can freely manipulate data to ensure successful backdoor injection while maintaining benign performance. Therefore, the motivation for studying clean-label backdoors in a training control setup is unclear.

2. The paper claims that PEFT algorithms struggle to successfully inject backdoors into LLMs. According to Table I, even dirty-label attacks (e.g., BadNets) using PEFT only achieve a 15.51% ASR on the SST-2 dataset. This observation contradicts recent literature on LLM backdoors [7, 8, 9]. For example, [7] reports successful backdoor injection into LLMs using QLoRA, and [8] proposes a fine-tuning method similar to PEFT that achieves effective backdoor injection. Can the authors clarify the reasons behind these contradictory findings?

3. One of the main advantages of W2SAttack is its ability to inject backdoors into models that cannot be trained using full-parameter fine-tuning due to computational constraints. Therefore, it would strengthen the paper if the authors included results from applying W2SAttack to larger open-source LLMs, such as Llama-2-70B or Mixtral-8x7B. This would further support the argument for the proposed attack.

4. Another point of concern is that the paper focuses primarily on LLM discriminative tasks, such as sentiment classification, whereas LLMs are now predominantly used for generative tasks. Recent works have also explored backdoors in LLMs for generative tasks [10, 11]. It would be valuable if the authors extended their proposed attack to generative tasks to determine if the same observations hold in those contexts.

---

Reference 

[1] Liu, Yunfei, et al. "Reflection backdoor: A natural backdoor attack on deep neural networks." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16. Springer International Publishing, 2020.

[2] Barni, Mauro, Kassem Kallas, and Benedetta Tondi. "A new backdoor attack in cnns by training set corruption without label poisoning." 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.

[3] Zeng, Yi, et al. "Narcissus: A practical clean-label backdoor attack with limited information." Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 2023.

[4] Turner, Alexander, Dimitris Tsipras, and Aleksander Madry. "Clean-label backdoor attacks." (2018).

[5] Cheng, Siyuan, et al. "Deep feature space trojan attack of neural networks by controlled detoxification." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 2. 2021.

[6] Doan, Khoa, et al. "Lira: Learnable, imperceptible and robust backdoor attacks." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

[7] Huang, Hai, et al. "Composite backdoor attacks against large language models." arXiv preprint arXiv:2310.07676 (2023).

[8] Li, Yanzhou, et al. "Badedit: Backdooring large language models by model editing." arXiv preprint arXiv:2403.13355(2024).

[9] Li, Yige, et al. "Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models." arXiv preprint arXiv:2408.12798 (2024).

[10] Rando, Javier, and Florian Tramèr. "Universal jailbreak backdoors from poisoned human feedback." arXiv preprint arXiv:2311.14455 (2023).

[11] Hubinger, Evan, et al. "Sleeper agents: Training deceptive llms that persist through safety training." arXiv preprint arXiv:2401.05566 (2024).
1. The research topic is interesting and important to the community

2. The idea is novel and intuitive.

3. The paper is overall well-written and easy to follow
This paper introduces W2SAttack, a method for injecting clean-label backdoors into LLMs. The approach stems from the observation that successfully injecting clean-label backdoors during fine-tuning becomes challenging when using Parameter Efficient Fine-Tuning (PEFT) algorithms. The authors analyze the limitations of PEFT from the information theory perspective and propose Weak-to-Strong Attack (W2SAttack) to enhance attack effectiveness under PEFT. Inspired by teacher-student knowledge distillation, W2SAttack first injects backdoors into a smaller teacher model using full-parameter fine-tuning. It then transfers the backdoor knowledge to a larger student model through PEFT, incorporating feature alignment loss terms during the distillation process to support the backdoor learning. Evaluation results demonstrate that W2SAttack can effectively inject various types of backdoors into LLMs using PEFT.
1. The threat model combining clean-label backdoor attacks with training control is counter-intuitive and lacks practical value.

2. The observation that PEFT cannot successfully inject backdoors is inconsistent with findings in recent literature.

3. The paper lacks evaluation on larger LLMs to demonstrate the scalability and effectiveness of the proposed method.

4. The paper lacks evaluation on generative tasks, which are a major use case for LLMs today.
Yes
No ethics review needed.
The definition of ASR(f(x^' )_peft) in Obj. 1 needs further clarification.
	What is the definition of Z_t? Additionally, the author needs to further explain why I(Z_t;Y) is related to backdoor features.
	I didn’t find the implementation details for Eq. 9 and 10, particularly for Eq. 9. Thus I have a concern about their correctness. Please provide more details.
	The author said they use the clean-label backdoor attack. Why don’t use the poison-label backdoor attack? Is there any difference between those two attacks in your method? Please clarify. Besides, the author should provide the details for attack, such as target label to solve my concern.
	I wonder if continuously increasing the number of poisoned samples would improve the attack success rate in the PEFT setting?
	The method of inserting triggers also affects the attack success rates. The author needs to further explain the implementation details of BadNet and InSent, as well as the SynAttack algorithm.
	The caption for Fig. 3 should provide a detailed description of the motivation for each subfigure.
	What is the meaning of ‘Efficient-tuning’ in Tab. 5?
	Also a concern about reproducibility, in Tab. 9, there is few details provided in terms of defense. For example, which trigger you used?
1.	The article proposes a counter-intuitive but effective framework, that is, using small models as teachers and large models as students. This makes me think it's quite novel
2.	The writing is fluent and clear, easy to understand
This paper proposed a method called W2SAttack. The author claims 1. that full-parameter fine-tune for achieving backdoor attack is not feasible due to high occupied VRAM 2. PEFT such as LoRA causes poor performance.
To address the posed problem, the author proposed the W2SAttack. They use the PEFT for smaller LLM first, and then set it as the teacher model to distill the larger LLM.
The results showed that the method can significantly reduce the computational cost.
Some weaknesses can be found in the Questions.
Yes
No ethics review needed.
More Explanation:
	Figure 1's y-axis should have a label; I was confused about its unit and what it represents.
	CA and ASR should be clearly mentioned in the caption.
	During the process of poisoning the teacher model, the authors added an additional linear layer. Is this layer necessary? Equation 4 requires further modification. What are the impacts of the teacher model on backdoor attacks?
	The expression of the attacker's Objective 1 indeed requires additional explanation. The authors have noted in their third stage pilot study that deploying effective backdoor attacks using the PEFT algorithm is challenging. However, Objective 1 suggests that ASR(f(x^' )_peft)≈ASR(f(x^' )_fpft), a statement that seems to contradict the earlier assessment, which requires further explanation.
Experimental Section:
	Is the caption for Figure 4 correct? The authors discuss the impact of different trigger lengths on backdoor attacks in the experimental analysis section, therefore this part needs to be revised.
	Compared to the BadNets backdoor attack, the backdoor attack algorithms based on InSent or SynAttack seem to achieve more desirable effects. Could the authors provide further analysis of the reasons for this, or present a more detailed analysis?
	Although the W2SAttack algorithm can guide the model to learn backdoor features, it requires the design of an additional teacher model. Existing experiments have only analyzed the effectiveness of the backdoor attack, but lack necessary analyses of communication costs, such as the training costs induced by changes in updatable parameters, which are essential for assessing the feasibility of the algorithm.
1.	Enhancing the effectiveness of backdoor attacks targeting the PEFT algorithm is a worthwhile research problem.
2.	The authors design an effective backdoor attack algorithm that saves computational resources compared to full-parameter fine-tuning.
3.	Overall, the presentation is clear, and the experiments are comprehensive. The details are clear.
The preliminary experiments in this paper discovered that the PEFT, which updates only a small number of model parameters, hardly implements backdoor attacks effectively. Based on these findings, the authors proposed a weak-to-strong backdoor attack algorithm targeting PEFT, named W2SAttack. They leverage a small-scale teacher model to facilitate the student model's learning of backdoor features, thereby enhancing the effectiveness of the backdoor attack.
Some aspects are not clear, see the questions section.
Yes
No ethics review needed.
1. What is the difference between the full-parameter fine-tuning of a small model in knowledge distillation and the full-parameter fine-tuning of a backdoored small model claimed in this paper?
1. PEFT in inheriting backdoors and learning backdoors using PEFT is a key research area targeting the security of LLMs.

2. Extensive experiments proved the feasibility of the attack.
This paper proposes a backdoor attack from weak to strong based on feature alignment-enhanced feature distillation. Extensive experiments show the superior performance of W2SAttack targeting PEFT on classification tasks across four language models, four backdoor attack algorithms, and two different architectures of teacher models.
**1: Motivation**
- The authors claim that LLMs cannot learn the backdoor under PEFT, but as far as I know, a lot of work reveals the vulnerability of PEFT against LLMs, e.g., references [1-2]. In addition, using LoRA (e.g., r=4) to implant a backdoor on NLU and NLG tasks,  the ASR is very easy to reach 100%.  

- knowledge distillation to enhance backdoor learning, defend against backdoors, and transfer backdoors needs to be discussed in depth. Therefore, related work is a crucial part of the main body. This helps to understand that the work enhances backdoor learning in the form of distillation, and the final release is an E2E backdoored model.

**2: Overclamming and misleading statement**

- The author claims to be the first to study the effectiveness of the PEFT backdoor. In fact, there are many works in this field, referring to references [1-3].

- When using Onion against W2SAttack, the results barely drop. However, Onion's effectiveness on word-level attacks can make attacks drop to at least around ASR of 50%.

**3: Presentation**

- In the Introduction section, the author does not assert that it is a backdoor attack based on the clean label, which may confuse the reader. 

- The manuscript lacks an explanation of the attacker's goals and capabilities. As I understand it, despite being a backdoor to clean labels, it requires poisoning the training set. Therefore, this assumption must be clarified in knowledge distillation or it will become impractical.

- Related work and experiment details are introduced in the appendix. The main body is not self-contained. 

- E should be corrected to $\mathbb{E}$ in Equation 3, 5, and 6.

**Reference**

[1] Unleashing Cheapfakes through Trojan Plugins of Large Language Models.

[2] A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning

[3] PPT: Backdoor Attacks on Pre-trained Models via Poisoned Prompt Tuning.
Backdoor Attacks with Knowledge Distillation
@inproceedings{
anonymous2024backdoor,
title={Backdoor Attacks for {LLM}s with Weak-To-Strong Knowledge Distillation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=29LC48aY3U},
note={under review}
}
Despite being widely applied due to their exceptional capabilities, Large Language Models (LLMs) have been proven to be vulnerable to backdoor attacks. These attacks introduce targeted vulnerabilities into LLMs by poisoning training samples and full-parameter fine-tuning. However, this kind of backdoor attack is limited since they require significant computational resources, especially as the size of LLMs increases. Besides, parameter-efficient fine-tuning (PEFT) offers an alternative but the restricted parameter updating may impede the alignment of triggers with target labels. In this study, we first verify that backdoor attacks with PEFT may encounter challenges in achieving feasible performance. To address these issues and improve the effectiveness of backdoor attacks with PEFT, we propose a novel backdoor attack algorithm from weak to strong based on feature alignment-enhanced knowledge distillation (W2SAttack). Specifically, we poison small-scale language models through full-parameter fine-tuning to serve as the teacher model. The teacher model then covertly transfers the backdoor to the large-scale student model through feature alignment-enhanced knowledge distillation, which employs PEFT. Theoretical analysis reveals that W2SAttack has the potential to augment the effectiveness of backdoor attacks. We demonstrate the superior performance of W2SAttack on classification tasks across four language models, four backdoor attack algorithms, and two different architectures of teacher models. Experimental results indicate success rates close to 100% for backdoor attacks targeting PEFT.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Backdoor Attacks Large Language Models Knowledge Distillation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/caaa9bf12a1cca2444e903263c94276c67523450.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/fdb532cb398071926b09947a1330895733d892db.zip
Backdoor Attacks for LLMs with Weak-To-Strong Knowledge Distillation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission