Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment
privacy;LLM alignment;rlhf
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
@misc{
feng2024exposing,
title={Exposing Privacy Gaps: Membership Inference Attack on Preference Data for {LLM} Alignment},
author={Qizhang Feng and Siva Rajesh Kasa and SANTHOSH KUMAR KASA and Hyokun Yun and Choon Hui Teo and Sravan Babu Bodapati},
year={2024},
url={https://openreview.net/forum?id=p7vItQ3OfD}
}
Large Language Models (LLMs) have seen widespread adoption due to their remarkable natural language capabilities. However, when deploying them in real-world settings, it is important to align LLMs to generate texts according to acceptable human standards. Methods such as Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO) have enabled significant progress in refining LLMs using human preference data. However, the privacy concerns inherent in utilizing such preference data have yet to be adequately studied. In this paper, we investigate the vulnerability of LLMs aligned using two widely used methods - DPO and PPO - to membership inference attacks (MIAs). Our study has two main contributions: first, we theoretically motivate that DPO models are more vulnerable to MIA compared to PPO models; second, we introduce a novel reference-based attack framework specifically for analyzing preference data called PREMIA (\uline{Pre}ference data \uline{MIA}). Using PREMIA and existing baselines we empirically show that DPO models have a relatively heightened vulnerability towards MIA.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Qizhang_Feng1 ~Siva_Rajesh_Kasa1 ~SANTHOSH_KUMAR_KASA1 ~Hyokun_Yun1 ~Choon_Hui_Teo1 ~Sravan_Babu_Bodapati1
Qizhang Feng Siva Rajesh Kasa SANTHOSH KUMAR KASA Hyokun Yun Choon Hui Teo Sravan Babu Bodapati
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
privacy LLM alignment rlhf
I certify that there is no acknowledgement section in this submission for double blind review.
feng|exposing_privacy_gaps_membership_inference_attack_on_preference_data_for_llm_alignment
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission