Evading Data Contamination Detection for Language Models is (too) Easy
large language models;model evaluation;malicious actors
Yes
No ethics review needed.
Please refer to the weakness part about the paper structure, technical depth, and novelty.
1. The analysis of existing methods is reaonsable. The paper starts the discussion on categorizing existing paper with a figure and the categorization is reasonable. 
2. The selection of base models and benchmarks is representative and comprehensive.
This paper focuses on the data contamination problem with an analysis of existing frameworks. The paper categorizes existing frameworks in 4 categories and reveals their weaknesses by a detailed study. The paper proposes a new attack strategy based on the analysis and observation.
The paper lacks novelty and technical depth, and the presentation of the paper is hard-to-follow. 
1. The paper mainly studies the existing methods, however, it lacks a detailed discussion on the proposed attack strategy and the reasonableness of such a strategy. As a reader, I cannot fully understand the main design of the strategy proposed after the analysis (In Sec 5.1 I suppose). It's hard to understand the method with a short paragraph and the paper lacks a formulation or figure for the strategy. 2. The experiments just focus on evaluating existing methods and it's hard to identify the section proving the effectiveness of the proposed strategy. The experimental design doesn't the support the claim of the paper and lacks crucial experimental results. 
3. The organization of the paper is dispersive with no key idea. The presentation is not coherent. After discussing the experiment results, the paper switched to a discussion on section 7, but such a discussion is not related to the experiment results presented before. This section sounds like a "related work" section.
Yes
By highlighting potential methods to evade current data contamination detection methods, the authors seek to promote awareness and encourage the development of more secure and reliable evaluation frameworks. While the paper aims to strengthen detection methods, these findings could be misused if taken out of context. Therefore, an ethics review may be beneficial to ensure the findings are communicated and applied responsibly.
Yes, Privacy, security and safety
- Is the performance across different benchmarks consistent? Could you provide a more detailed breakdown of per-benchmark performance?
- The paper notes that the limited sample size of the contaminated set contributes to increased variance in results. Would it be feasible to increase the sample size of the contaminated dataset to reduce this variance and enhance result stability?
- The paper introduces a novel categorization of model providers based on their data contamination practices.
- Provide a simple yet effective contamination technique to evade detection, highlighting limitations in current detection methods.
- The paper is well-organized, with each section building on the previous one. Key concepts like contamination types and detection vulnerabilities are presented accessibly, making the research easy to understand and follow.
This paper looks into the vulnerabilities and reliabilities of current data contamination detection methods. The authors categorize model providers based on their contamination practices, emphasizing malicious actors could intentionally contaminate models by rephrasing the benchmark data, thus evading detection while boosting model scores. Through experiments across several models and benchmarks, the authors show that the rephrasing technique could increase performance while bypassing existing detection methods.
The paper demonstrates the vulnerability of current detection methods to rephrasing attacks but does not explore potential defenses. Without suggestions for addressing this vulnerability, the work may feel incomplete.
Yes
No ethics review needed.
- Does the EAL technique circumvent all detection methods mentioned in A Taxonomy for Data Contamination in Large Language Models, including those that do not require pre-training data?
- Method is simple and easy to understand.
- The topic is highly relevant to the current challenges in assessing the performance of large language models.
The paper presents a examination of the vulnerabilities in current benchmarking practices for LLMs. They introduce Evasive Augmentation Learning (EAL), a technique that rephrases benchmark samples to boost performance on benchmarks and evade current detection methods.
- Unclear definition on "Definition 3"
- The distinguish between openly malicious and evasively malicious is somewhat problematic. It's hard to find realistic usage when applying the open malicious scenario to any of the attackers. Openly malicious is easily getting rid of by the other detection method listed.
- Sections 4 and 5 would benefit from streamlining. There is some repetition and unnecessary content that could be condensed to improve clarity and conciseness.
- Although the paper states that code is provided, it does not include detailed descriptions of the processes for data rephrasing and contamination insertion, particularly regarding how rephrased data is generated.
- The authors overclaimed the contribution. Failure to consider the technical feasibility and applicability of updating baseline data in real-world applications, resulting in proposed attack methods that are more suitable for closed environments and may not work in open and real-time updating environments
Yes
No ethics review needed.
Please see the comments above.
+ This work studies a valid topic -- detecting more evasive attack to endanger LLM safety.
+ This work summarizes many attacks and detection efforts.
This work aims to propose an evasive attack to bypass potential detection. The author first summarizes existing attacks, while studying potential detection strategies using metadata, reference models, semantic-preserving transformations, and different accessibility. Then, the author proposed that rephrasing can help to increase evasiveness.
- Many settings are unclear: (1) For "contamination," there is no definition of the threat model regarding its knowledge, capabilities, or concrete examples in text-based attacks. (2) It is also unclear how the attacker influences benchmarks, what the adversarial samples look like, and what the adversarial objectives are. (3) Furthermore, defense settings are also unclear about the role of detector/defender and what actions can the defender take to eliminate poisoning? The author is suggested to add those information about both threat model and defender's strategies.

- The paper uses overly complicated wording, which creates barriers to understanding. Specifically: (1) "Contamination" is typically used as "poisoning." (2) The "actor" is generally referred to as "attacker" or "adversary." The "actor" typically links to specific roles (such as specific cyber threat patterns). (3) "De-Contamination" is rarely used. An alternative term is "mitigation." (4) "verbatim" could be "word-wise." (5) "Sample-level" could be more accurately expressed as "sentence (or word, token)-level." (6) It is inaccurate to use "Problematic Data Contamination" in Definition 3, as that definition is talking about possibility of injecting poisoning data D'. 

- The author did not clearly explain why pre-training requires benchmarking data. The author probably means "public" or "crowdsourced" data.

- The paper over-claimed the contribution "We define four (de-)contamination settings..." where the settings are summarized, not defined in this paper.

- The role of model providers is unclear. In section 3, model providers are both attacker and defender. The author is suggested to distinguish model providers with adversaries (the "actor").

- The paper has poor structure and incomplete content: (1) There is no related work section. (2) The author did not detail the proposed rephrasing attack, such as what prompts or techniques are used for rephrasing. (3) Definition 4 simply repeats definition 1 and 2. (4) The Definition 5 should be highlighted with rigorous formulation with later technical design addressing it. (5) It would be better to introduce "TPR" (true positive rate) and "FPR" (false positive rate) in Figure 1.

- The contribution is not significant. The author simply uses "rephrasing," which lacks strong intuition and motivation for its usefulness.

- The experiments are simple, and the settings are problematic. Specifically: (1) the author mentions several detection strategies but does not provide corresponding experiments to address those detections. (2) The paper also fails to clarify how evasiveness is guaranteed -- Is it measured by lower TPR by detection methods? (3) No baseline attacks. (4) The author is suggested to discuss the trade-off between attack effectiveness and evasiveness.

- Many sentences seem GPT-generated. It is fine to use GPT to polish writing. However, the original writing maybe logically inconsistent and unclear, so GPT alone cannot produce better logic.
We analyze current contamination detection methods and find a significant vulnerability in their assumptions that can be easily exploited by malicious actors.
@inproceedings{
anonymous2024evading,
title={Evading Data Contamination Detection for Language Models is (too) Easy},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Nk1MegaPuG},
note={under review}
}
The benchmark performance of large language models (LLMs) has a high impact on their popularity and is thus of great importance to many model providers. However, the reliability of such benchmark scores as a measure of model quality gets compromised if the model is contaminated with benchmark data. While recent contamination detection methods try to address this issue, they overlook the possibility of deliberate contamination by malicious model providers aiming to evade detection. We propose a categorization of model providers based on their (de)contamination practices and argue that malicious contamination is of crucial importance as it casts doubt on the reliability of public benchmarks. To study this issue more rigorously, we analyze current contamination detection methods based on their assumptions. This analysis reveals a significant vulnerability in existing approaches: they do not account for rephrased benchmark data used during training by malicious actors. We demonstrate how exploiting this gap can result in significantly inflated benchmark scores while completely evading current detection methods.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
large language models model evaluation malicious actors
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/25521348353a653c06067b0032ed8b485cf60e72.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/1530125ef4f19d679aebc06f7405df3fc8b1e2d1.zip
Evading Data Contamination Detection for Language Models is (too) Easy
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission