Optimizing Adaptive Attacks against Content Watermarks for Language Models
watermarking;language models;robustness;adaptive attacks
Yes
No ethics review needed.
See weaknesses.
1. The proposed adaptive attack reveals the potential vulnerability when attackers are stronger, potentially facilitating the community in building more robust watermarks.

2. It experiments with diverse settings.
This work proposes an adaptive attack to remove watermarks by fine-tuning a paraphraser. It shows that when attackers know the watermark method, they are able to achieve stronger evasion attacks.
1. **Unclear optimization formulation.** Why do you formulate robustness for attack optimization? The robustness of what (the surrogate model or the target model)? Are the paraphraser and the surrogate model the same model (given line 136: *"Our attacker can access such open-weight surrogate models and use them for paraphrasing text."*)?

2. **The algorithm is confusing.** To obtain the paraphraser without directly optimizing equation 2, the authors propose using DPO to overcome this challenge, which requires a preference dataset. However, in Algorithm 1, generating this dataset requires the paraphraser. How do you obtain the paraphraser?

3. **The method lacks important technical details.** Isn't the attackers' objective to fine-tune a paraphraser to remove the watermark in texts generated by the target language model? You proposed using DPO for optimizing the paraphraser (i.e., equation 2) as stated in line 178-179, but the method only covers how the dataset is created, not how the paraphraser is optimized using DPO.

4. **Lack of important analysis of experiment results.** In Figure 3 (Left), why does the adaptive setting have better performance than the non-adaptive setting? This is counter-intuitive, as the adaptive setting involves training and testing the paraphraser on the same watermark scheme, which should yield the best performance. Also, How do you explain why the proposed method shows such good transferability to unseen watermarks?
Yes
No ethics review needed.
Please refer to the weaknesses part.
1. The paper is written in a clear and well-structured manner that makes it easy to follow the methodology and results.
2. The proposed attack model demonstrates superior performance compared to previous LLM-based approaches in removing watermarks.
This paper presents a rewriting attack model against LLM text watermarks. The authors train an LLM using DPO (Direct Preference Optimization) with collected preference data, achieving better attack performance compared to previous rewriting models.
1. The paper lacks comprehensive baseline testing, particularly against robust watermarking algorithms such as UnIgram (Zhao 2024 et al.) and SIR (Liu 2024 et al.).   Since these algorithms are currently generally considered more robust than algorithms like Exp and KGW, experiments on robustness attacks should be conducted on robust models.

2. Several issues exist in quality testing:
- The paper lacks sufficient details regarding the dataset used for watermarked text generation, particularly concerning the source of prompts used to generate data from the LLM. Without clear information about the prompt dataset used in this process, it becomes difficult to fully understand and reproduce their watermarking approach.
- The choice of Llama3-8B-Instruct for PPL calculation appears suboptimal when Llama3-70B resources are available.
- The quality metrics presented in Figures 3 and 4 lack clarity, with multiple metrics mentioned but only single values shown without explanation of averaging or weighting methods. Could you provide a clear explanation of how the single values in Figures 3 and 4 are derived from the multiple metrics mentioned?
- The quality test results appear questionable, particularly where Llama2-7B outperforms GPT-3.5, suggesting potential issues with data selection or evaluation methodology.

3. The paper lacks theoretical insights:
- There is no clear explanation of the mechanism by which the training approach effectively removes watermarks while preserving text quality, maybe more deep into the n-gram is required.  Could you provide an analysis of how your method affects n-gram distributions compared to the original text and other baselines.
- The paper fails to analyze why the proposed method achieves better performance than GPT-3.5, especially the text quality.

4. The evaluation would benefit from including stronger models like GPT-4 and GPT-4o to provide more comprehensive insights into the attack model's effectiveness.


References:

1. Provable Robust Watermarking for AI-Generated Text  ICLR 2024
2.  A semantic invariant robust watermark for large language models  ICLR 2024
Yes
No ethics review needed.
I have some deeper concerns that the authors should address:

1. The theoretical underpinnings of attack transferability need more rigorous examination. While the paper demonstrates impressive empirical results showing attacks transfer well to unseen watermarks, it lacks formal analysis of why this occurs. Could this suggest some fundamental limitation in current watermarking approaches? For instance, most current methods rely on statistical patterns in token distributions - perhaps this shared mathematical structure creates inherent vulnerabilities that sophisticated attacks can exploit. A theoretical framework explaining these transfer properties would be invaluable for designing more robust watermarks.

2. Why do these attacks transfer so well to unseen watermarks? The paper shows impressive empirical results but doesn't help us understand the underlying reasons. Could there be some common vulnerability across current watermarking approaches that we're missing?

3. The authors' threat model focuses primarily on demonstrating attack capabilities but pays insufficient attention to the defensive aspects of the security game. Given that watermarking is intended as a protective measure, understanding its limitations should naturally lead to exploring ways to strengthen it.

4. How well do these attacks generalize across different languages and domains?
I find this paper interesting as it challenges what many of us took for granted—the robustness of LLM watermarks. What sets it apart from previous work is its systematic approach to breaking watermarks. Instead of relying on manual perturbations or simple models, the authors frame watermark evasion as an optimization problem.

The most surprising finding, to me, is how effectively a modest 7B parameter model can be tuned to fool watermarks from much larger models. This raises serious questions about the fundamental security of current watermarking schemes. Are we perhaps building our defenses on shaky foundations?

The technical contributions are solid:

- This paper is well organized and easy to read
- The paper's key insight is transforming watermark robustness into an optimizable objective function, enabling systematic attack optimization rather than relying on heuristic approaches used in prior work. The strengths of the paper include:
- The preference-based optimization framework is clever as it overcomes the challenges of optimizing over discrete text while requiring minimal computational resources
- The approach is model-agnostic and can be applied to any watermarking method, making it a general-purpose attack framework
- Attack is  practical, requiring only 7 GPU hours and ~$10 of compute
- Novel finding that attacks can transfer to unseen watermarks, suggesting fundamental limitations in current approaches
This paper investigates the robustness of watermarking methods for Large Language Models (LLMs) against adaptive attacks. The authors propose methods to optimize attacks against known watermarking algorithms using preference-based optimization. Their key findings show that: (1) adaptive attacks significantly outperform non-adaptive baselines, achieving >96% evasion rate with minimal quality degradation; (2) attacks optimized against known watermarks remain effective against unseen watermarks; and (3) the optimization process requires limited computational resources. The work demonstrates important vulnerabilities in current LLM watermarking approaches.
While the empirical results presented in this paper are impressive, several important theoretical questions remain unanswered. Most notably, the authors demonstrate that their attacks transfer effectively across different watermarking schemes, but they don't provide a theoretical framework to explain why this happens. This gap in understanding is concerning - without knowing which properties of watermarking schemes make them vulnerable to transfer attacks, it becomes difficult to design more robust systems. The lack of formal bounds or guarantees on attack effectiveness also leaves us wondering about the fundamental limits of watermark robustness.


Another limitation of this work lies in its treatment of defensive measures. While the authors demonstrate impressive attack capabilities that can evade watermark detection with a>96% success rate using minimal resources, they offer little insight into potential countermeasures. Could watermarking schemes be made adaptive, perhaps updating their strategies based on observed attack patterns? Are there fundamentally different approaches to watermarking that might prove more robust? The paper's silence on these questions leaves practitioners with clear evidence of the problem but little guidance on potential solutions.
Yes
No ethics review needed.
- What exact settings for KGW were run, did the authors try to run more robust versions of the watermark? How does basic paraphrasing perform already perform so well?
- Can you give more qualitative insight as to why the approach works well. Are the notes from above (learns to avoid patterns) essentially what you observe qualitatively?
- See other individual points above.
- Stronger adversarial evaluations of LLM watermarks are a timely and important area of research for improvements in LLM watermarks. The reviewer agrees with the authors that existing work (with exceptions - see below) has focussed too little on this.
- The setup of the evaluation is realistic, using both surrogate models, an overall sensible text quality evaluation, and general black-box assumptions.
- Results indicate that learning on the preference data achieves overall higher evasion rates at higher text quality.
The paper proposes an adaptive (finetuned) paraphrasing adversary against several types of LLM watermarks. In particular, an adversary first collects samples of watermarked and paraphrased texts, building a preference dataset based on paraphrases that achieve watermark removal and maintain overall text quality (over a range of surrogate models and keys). With this DPO dataset the adversary then finetunes paraphrase LLM(s). The evaluation shows that this works for (and across) three watermarking scheme families, leading to a higher watermark removal rate while retaining better text quality.
- I will focus on Dist-Shift (KGW) and Exp (Aaronson), as they are arguably the most robust and well-known of the four tested schemes. The overall evasion rates, even for basic paraphrasing, seem very high to me. While my personal experience is more with KGW than EXP, the works that I have seen have significantly lower evasion rates (e.g. 0.18 at a FNR of 1e-3 on SelfHash (h=3, Llama 7B) in [1] Table 4 (a setting recommended in [3]) or in [3] itself in Fig.5 ~50% for LeftHash h=1) than the ones around 85% reported in Figure 5. This currently leaves two points unclear for me: (1) What are the exact Dist-Shift/KGW settings used by this work (as it seems rather weak) and (2) How to interpret the advantage an adversary gets from this as, e.g., in Figure 8 the gap of the finetuned models to GPT3.5 is already small (and might be even smaller with stronger zero-shot paraphrasers).
- Some intuitive explanation / qualitative explanation why one would expect this methodology should work for (e.g.) Dist-Shift. A priori, it is unclear why a paraphrasing model should be able to learn to essentially avoid green-list tokens for an unseen watermarking key as the key "randomly splits" the vocabulary into green and red. However, it makes sense as it seems to avoid repeating token structures observed in the input. Transferability between methods could then be explained by this observation being the driving factor across all methods. A qualitative experiment that highlights this and possibly other factors could improve the presentation of the approach and might lead to insight into, e.g., adaptive defenses.
- Unlike some claims in the paper, there is definitely work on stronger adversaries against LLM watermarks, e.g., [1], [2] which explicitly and implicitly assume knowledge about the watermarking algorithm.
### Typos/Nits
- What exactly is the tradeoff in L265: "Higher robustness leads to a higher computational burden for the attacker"? To me, it seems this is a direct consequence of the robustness.
- Instead of going very high in the FPR range (e.g., 10% in Figure 5), It would make more sense in the reviewer's opinion to ablate over low FPR regions as these are the ones where watermarks are most likely applied (e.g., both [1] and [3] also evaluate on 0.001).

[1] Jovanović, Nikola, Robin Staab, and Martin Vechev. "Watermark stealing in large language models." ICML (2024).

[2] Pang, Qi, et al. "No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices" _arXiv preprint arXiv:2402.16187_ (2024).

[3] Kirchenbauer, John, et al. "On the reliability of watermarks for large language models." ICLR 2024.
We propose methods to optimize adaptive attacks against content watermarks for language models and demonstrate the necessity to test robustness against adaptive attacks.
@inproceedings{
anonymous2024optimizing,
title={Optimizing Adaptive Attacks against Content Watermarks for Language Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RKQcJ1lXNT},
note={under review}
}
Large Language Models (LLMs) can be \emph{misused} to spread online spam and misinformation. Content watermarking deters misuse by hiding a message in model-generated outputs, enabling their detection using a secret watermarking key. Robustness is a core security property, stating that evading detection requires (significant) degradation of the content's quality. Many LLM watermarking methods have been proposed, but robustness is tested only against non-adaptive attackers who lack knowledge of the watermarking method and can find only suboptimal attacks. We formulate the robustness of LLM watermarking as an objective function and propose preference-based optimization to tune \emph{adaptive} attacks against the specific watermarking method. Our evaluation shows that (i) adaptive attacks substantially outperform non-adaptive baselines. (ii) Even in a non-adaptive setting, adaptive attacks optimized against a few known watermarks remain highly effective when tested against other unseen watermarks, and (iii) optimization-based attacks are practical and need limited computational resources of less than seven GPU hours. Our findings underscore the need to test robustness against adaptive attackers.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
watermarking language models robustness adaptive attacks
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/1040dbbef6ef885bc9fb6bf583ab17a0a8ac109b.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Optimizing Adaptive Attacks against Content Watermarks for Language Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission