Rethinking Adversarial Attacks as Protection Against Diffusion-based Mimicry
Generative Model; Diffusion Model; Adversarial Attack
Yes
There are potentially copyrighted images in the paper, which may or may not be subject to copyright issues.
Yes, Legal compliance (e.g., GDPR, copyright, terms of use)
Please see weakness for my concerns, where I have included details already.
The limitations of existing protection methods are indeed something worth highlighting, and the paper did a fair job in providing experimental supports to primary claims.
This paper raises two concerns against IP protection methods based on adversarial attacks to diffusion models, both based on the observation that PDMs (pixel space diffusion models) are more robust to adversarial attacks than LDMs (latent diffusion models).

concern 1. The adversarial attack-based IP protection methods are less effective for PDMs.

concern 2. Using PDM-based purification to preprocess "protected" images can also reduce the effectiveness of adversarial attack-based IP protection methods.

These two are the primary contributions of the submission.
There are a few weaknesses but I think they should be fixable:
1. Inaccurate position of their contributions among existing research: 
Adversarial robustness of diffusion models (or PDMs) and using diffusion purifications to mitigate adversarial attacks have already been investigated in existing research. While the authors have already cited DiffPure (Nie et al., 2022) in the submission, the usage of DiffPure-like techniques and their contributions are without a doubt outlooked in the story told. This should be fixed by acknowledging (in a much more prominent way) the contributions of existing work adversarial robustness of diffusion models (or PDMs) and using diffusion purifications to mitigate adversarial/other noises.

2. The quantitative results for the PDM-Pure part seem incomplete in a sense that only one metric, FID-score, was reported.
Since 3 other metrics were reported in your Quantitative Measurement of PGD-based Adv-Attacks for LDMs and PDMs (table 2), I believe the authors have no trouble finding more quantitative metrics for evaluating PDM-Pure. I would suggest authors to incorporate these as well.
Yes
No ethics review needed.
Overall, the research topic and experiments are good. I would suggest that the authors fix all typos and  focus more on (1) demonstrating convincingly that adaptive attacks are ineffective, and (2) discussing why PDM-Pure performs well. 
Additionally, there’s no need to spend excessive space explaining why adversarial attacks designed for LDMs are less effective for PDMs, as this is fairly intuitive.
This paper focuses on an intriguing topic regarding attackers who aim to target pixel-based diffusion models and attempt to eliminate protective perturbations. It highlights that tools relying on adversarial perturbations cannot consistently safeguard artists from the misuse of generative AI.

Additionally, the authors conducted several adaptive attacks, transitioning adversarial attacks from Latent Diffusion Models (LDMs) to Pixel Diffusion Models (PDMs). They also introduce their straightforward yet effective method, PDM-PURE. 


They also conducted various experiments on advanced protection methods, such as Photoguard, Glaze, and Mist, which indicate that these methods cannot effectively prevent the misuse of generative AI.
This paper primarily focuses on adversarial attacks targeting diffusion models. The authors first discover that previously effective attacks on latent diffusion models (LDMs) are no longer successful against pixel-based diffusion models (PDMs). Furthermore, the authors demonstrate that PDMs can serve as an effective off-the-shelf purifier, capable of eliminating adversarial patterns generated by LDMs. This preservation of image integrity effectively removes protective perturbations of various scales encountered in applications like Glaze and Mist.
To enhance the overall interest of the story, I believe addressing the following points could improve this work:

1. The paper is not well-prepared, with many typos, formatting errors, and grammar issues:

   - In lines 15 and 436, there's an extra space before the comma.
   - In line 19, "PDM" is not well-defined initially, yet it is repeatedly redefined in several places, such as in lines 163 and 477.
   - There are typos in lines 266 and 267: "to" and ")". 
   - In line 265, where it says `$l_\infty > 150/255$`, do you mean $\delta$?
   - Typo in line 281: "should."
   - Typo in line 295: "decode."
   - In line 469, `t*`; in line 471, `??`.


2. Some techniques are not well-explained:
   - While I understand that attacks designed for latent space are less effective in pixel space, the authors seem to spend excessive space discussing this point (Section 4 already addressed this point, but Section 6.2 reiterates it.). It’s fairly intuitive that different optimization losses lead to this limitation. What would be more insightful is a deeper exploration of adaptive attacks in pixel space. Here are some specific questions:
  
     1. Could you elaborate on your two categories of attacks (lines 247-248)? From my understanding, a reasonable approach might be to adapt the loss in Equation 1 to match the loss function used in pixel-based models. Additionally, it could help to incorporate information from intermediate layers, run EOT multiple times, and use these gradients to update adversarial images. Based on the current description, it’s unclear if the adaptive attack implemented here is sufficiently strong.
      2. The authors use Figure 2 to explain why pixel space adversarial attacks are not effective, particularly due to the large overlap. However, for traditional adversarial attacks on ResNet models using the ImageNet dataset, the adversarial examples are generated through pixel-level updates, no? This method has also proven to be very successful. any difference here?
     3. In line 318, what do you mean by "domain shift"?  and why?

   - The explanation of PDM-Pure is also insufficiently detailed (taking up less than half a page on page 7). Specifically:
     
     1. In line 362, the authors state that "the adversarial pattern should be removed," but the rationale behind this is unclear.
     2. In line 465, it’s mentioned that "LDMs cannot be used to purify the protected images." This seems like an overstatement; as far as I know, LDM (Stable Diffusion) has been used in prior work[1] to successfully attack Glaze.


[1] Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI. arxiv: 2406.12027
Yes
No ethics review needed.
Would you also add a code to this framework, please?
- Novel Contribution: The paper identifies a critical oversight in the current understanding of adversarial attacks on diffusion models by focusing on PDMs, which have been largely neglected in prior research. 

- Robust Experimental Validation: The authors conduct comprehensive experiments across various architectures and datasets, demonstrating the superior robustness of PDMs compared to LDMs against adversarial perturbations. 

- Practical Implications: The introduction of the PDM-Pure framework offers a practical solution for purifying images from adversarial attacks, which could have significant implications for protecting intellectual property in digital media. 

- Clear Presentation: The paper is well-structured and clearly presents its findings, making it accessible to readers with varying levels of expertise in the field.
The paper explores the robustness of pixel-space diffusion models (PDMs) against adversarial attacks, contrasting them with latent diffusion models (LDMs). The authors highlight a significant gap in existing literature, which predominantly focuses on LDMs while neglecting PDMs. Through extensive experimentation, they demonstrate that traditional adversarial attack methods fail to effectively compromise PDMs, suggesting that these models possess a higher degree of robustness. The authors propose a new framework, PDM-Pure, which utilizes PDMs as a universal purifier to eliminate adversarial perturbations generated by LDMs, thereby enhancing image integrity.  
In conclusion, the paper not only fills a critical gap in the literature but also provides actionable insights that could influence future research and applications in the field. Its novel contributions and practical implications make it a valuable addition to the conference proceedings.
- Limited Scope of Experiments: While the experiments are extensive, they may not cover all potential adversarial attack methods or variations in model architectures that could further validate the findings.  

- Assumption of Robustness: The conclusion that PDMs are universally robust may require further exploration under different conditions or with more sophisticated attack strategies that were not tested in this study. 

- Lack of Discussion on Limitations: The paper could benefit from a more thorough discussion of the limitations of their approach and potential scenarios where PDMs might still be vulnerable to attacks.
Yes
No ethics review needed.
- How does the technical similarity between your paper and prior works such as GrIDPure and DiffPure impact the novelty and contribution of your study? 

- PDMs are also neural networks after all. What might account for their empirical robustness to adversarial perturbations?

- Can the adversarial robustness of PDMs in generative tasks also be applied to classification tasks? I recently came across studies by Chen et al. [2,3] (I did not go into the details of these papers as they are full of theories and mathematical proofs), which propose that diffusion models can serve as certifiably robust classifiers, but I'm not sure whether this is strongly connected to your discovery on PDMs. Could the authors share their perspective on this potential crossover?

- I think another limitaion is that PDMs may not scale as good as LDMs, making the potential of PDM-Pure for large scale purification difficult. Can the authors share their opions and comments on this aspect?


Ref:

[2]: Chen et al. Robust Classification via a Single Diffusion Model. ICML 2024.

[3]: Chen et al. Diffusion Models are Certifiably Robust Classifiers. NeurIPS 2024.
- The motivation for testing protective perturbations designed for LDMs on PDMs is interesting and novel. This paper's discoveries and results are also fresh and insightful.

- The preliminary results challenge the common belief that DMs are highly susceptible to adversarial attacks, offering new perspectives for the community.

- This paper tries to evaluate PDMs against adaptive attacks and the attack from the latest work (Shih et al., 2024), which should be encouraged.

- The paper is generally well-written and motivated. The proposed method is straightforward, effective, cleverly designed, and clearly explained.

- The experiments cover a wide range of cutting-edge diffusion models, with impressive results that likely open new avenues for disruption-based protection against diffusion-driven editing and mimicry.

- The authors acknowledged the trade-off between purification effectiveness and image quality, which is commendable.
This paper investigates the protective protection against diffusion-driven editting/mimicrying. The protective protection in previous works is generally adversarial perturbations designed against latent diffusion models due to the huge impact of Stable Diffusion model families. Following the findings of previous literature that discovered current perturbations mainly attacks the VAE part of the LDM, this paper conducts experiments on attacking PDMs and find they are (empirically) surprisingly more robust than LDMs, even under EOT-based adaptive attacks and recently proposed adaptive attacks. Based on this understanding, the paper proposes a simple yet effective defense (or attack against defensive protection) named PDM-Pure, which directly uses pre-trained PDMs and SDEdit to purify a given image and remove the injected noises. The results on several SOTA diffusion models and attacks demonstrates the effectiveness of PDM-Pure.
- While I appreciate the effort to evaluate PDMs against adaptive attacks and the recent attack proposed by Shih et al. (2024), the results in Figures 3 and 4 seem to be a rudimentary case study rather than a comprehensive experiment. Thus, the claim that PDMs are also robust against adaptive attacks is not very convincing. Moreover, the claim in Fig. 4 that Shih et al.'s attack needs perturbation buget of $\ell_{\infty}>150/255$ gives a sense that this attack is not truely adaptive or not correctly applied, because at the perturbation budget of this level, any image can be converted into a gray image (i.e., (127,127,127), or #7f7f7f in Hex, at every pixel), making it impossible for the model to identify original features. Besides, it would be helpful for the authors to clarify the experimental setup used in these figures (i.e., directly use SDEdit to noise the protected image and denoise it, if I understand correctly). Because at first glance, the images appear to be merely restored rather than edited (e.g., in typical image editing, one might expect transformations like a sunset being altered to a sunrise).

- Although the authors provide a reasonable explanation for why PDMs seem more robust than LDMs, it remains unclear why PDMs demonstrate such strong empirical robustness. Since PDMs are still neural networks, one might expect them to also be vulnerable to adversarial attacks. I recommend that the authors further analyze and hypothesize the reasons behind this robustness to enhance the reliability of their findings and deepen our understanding of the method’s mechanisms. More interestingly, recent literature have discovered the potential of diffusion models to be applied for classification tasks [1]. Does that mean we can obtain a zero-shot and adversarially robust classifier for free? I encourage the authors to discuss this possibility and any challenges they foresee.

- From a technical perspective, the contributions in this work seem similar to those in GrIDPure and DiffPure (specifically similar to DiffPure, whose idea is also use noising-denoising nature of DMs to purify adversarial noises). A more detailed explanation and comparison would help to clarify the paper’s uniqueness and value.

The reviewer would like to actively participate in the disucssion and will adjust the rating accordingly.

Minor:

- Figure 3: PDM should robustness -> PDM shows robustness 
- L252-257: please consider unifying the abbreviation of EOT
- L267: as shown in the last block of Figure 4) -> as shown in the last block of Figure 4
- L296: it barely work -> it barely works

Ref:

[1]: Li et al. Your diffusion model is secretly a zero-shot classifier. ICCV 2023.
@inproceedings{
anonymous2024rethinking,
title={Rethinking Adversarial Attacks as Protection Against Diffusion-based Mimicry},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=tiJzOop4u6},
note={under review}
}
Diffusion models have demonstrated an remarkable capability to edit or imitate images, which has raised concerns regarding the safeguarding of intellectual property. To address these concerns, the adoption of adversarial attacks, which introduce adversarial perturbations that can fool the targeted diffusion model into protected images , has emerged as a viable solution. Consequently, diffusion models, like many other deep network models, are believed to be susceptible to adversarial attacks. However, in this work, we draw attention to an important oversight in existing research, as all previous studies have focused solely on attacking latent diffusion models (LDMs), neglecting adversarial examples for diffusion models in the pixel space (PDMs). Through extensive experiments, we demonstrate that nearly all existing adversarial attack methods designed for LDMs, as well as adaptive attacks designed for PDMs, fail when applied to PDMs. We attribute the vulnerability of LDMs to their encoders, indicating that diffusion models exhibit strong robustness against adversarial attacks. Building upon this insight, we find that PDMs can be used as an off-the-shelf purifier to effectively eliminate adversarial patterns generated by LDMs, thereby maintaining the integrity of images. Notably, we highlight that most existing protection methods can be easily bypassed using PDM-based purification. We hope our findings prompt a reevaluation of adversarial samples for diffusion models as potential protection methods.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Generative Model; Diffusion Model; Adversarial Attack
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ba8c6dd49f03614be1efcf0531f1bb633aa16259.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Rethinking Adversarial Attacks as Protection Against Diffusion-based Mimicry
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission