Prompt-Independent Safe Decoding to Restrain Unsafe Image Generation for Text-to-Image Models against White-Box Adversary
Text-to-image generation;AI security;Model compliance
Yes
No ethics review needed.
Please refer to the weakness section.
1. This paper addresses a crucial safety issue: defending T2I models against white-box adversaries. As T2I models become more popular and widely deployed, this topic is increasingly important.

2. The paper presents a novel and intriguing idea.

3. The content is well-structured and easy to follow.
This paper introduces a novel defensive solution called Patronus, designed to prevent T2Is from generating unsafe content. Patronus integrates an output moderator directly within the T2I's decoder. It guides the image decoder to perform conditional decoding based on safety features, transforming unsafe input features into zero vectors.
While the paper introduces 'Patronus' as a novel defensive mechanism, the evaluation section of this paper is too weak, making the effectiveness of the proposed method unclear, which is essential for achieving a higher-scoring paper. 

1. Lacking Evaluation Results on SOTA adversarial attack for T2I. The authors should assess the proposed defense against leading Text-to-Image adversarial attacks, such as MMA-Diffusion [c0]. This attack has been reported to outperform I2P, posing a significant threat to T2I models. Evaluating the defense's effectiveness by reporting the Attack Success Rate after implementing Patronus would demonstrate its generalizability and robustness.

[c0] https://arxiv.org/abs/2311.17516

2. The related work section overlooks a key contribution, [c1], which fine-tunes a text decoder to defend against adversarial prompts. In contrast to this approach, the present work focuses on fine-tuning the image decoder. A discussion of [c1] should be included in the related work, highlighting this crucial difference in methodology and its potential implications. This comparison will provide a more complete context for the current work and clarify its unique contribution.
 
[c1] https://arxiv.org/pdf/2403.01446

3. Lacking comparison with close-related baselines.  The authors assert that existing content moderators "can be easily removed by white-box adversaries" and have other drawbacks (lines 037-041). However, it's unclear how effectively the proposed "Patronus" addresses these issues. To demonstrate its superiority, it is recommended that the authors compare their method with existing content moderation solutions such as LlamaGuard [c2] and OpenAI-Moderation [c3]. This comparison would highlight the effectiveness of Patronus in overcoming these challenges.

[c2] https://arxiv.org/abs/2312.06674

[c3] https://arxiv.org/pdf/2208.03274

4. Limited Evaluation Metrics: Relying solely on "CLIPScore" offers an incomplete view of the proposed solution's performance. The error bar in Figure 3 nearly obscures the performance gain. A more comprehensive evaluation should include metrics such as Attack Success Rate (ASR), Area Under the Receiver Operating Characteristic Curve (AUROC), and False Positive Rate (FPR), as recommended in [c1], [c2], and [c3]. These metrics provide a clearer understanding of the solution's impact on both normal and unsafe use cases.

5. Absence of Adaptive Attack Evaluation: The evaluation doesn't consider the critical scenario of adaptive attacks, where the attacker possesses full knowledge of both the T2I model and the implemented safeguards. Evaluating the defense's resilience against adaptive attacks is crucial for understanding its real-world effectiveness. The authors can follow the adaptive attack design principles suggested in [c4] to conduct experiments.

[c4] Tramer, F., Carlini, N., Brendel, W., & Madry, A. (2020). On adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33, 1633-1645.
Yes
No ethics review needed.
Please see the above weaknesses.
This paper aims to solve a challenging problem. For an open-source model, malicious fine-tuning to remove the safety alignment is a key challenge. 

The paper follows a good formulation.

Evaluation is conducted on multiple datasets, and multiple baseline alignment methods are compared.
This paper tries to propose a method to align text-to-image model to prevent unsafe image generation. The idea is to fine-tune the decoder and diffusion model (U-Net). Key challenge for alignment is that if the model is open-source, an attacker may fine-tune the model to remove the alignment. This paper aims to solve this challenge via aligning the model in a specific way. The alignment itself is a fine-tuning process. Evaluation is performed on several datasets and some malicious fine-tuning.
The most important weakness is that the reviewer is not convinced the proposed method solves the challenge. It is not clear why the proposed alignment method can survive malicious fine-tuning. It's not clear to why the proposed method is different from baselines in terms of resistance against malicious fine-tuning. 

Evaluation metrics are too simple. Mainly CLIP is used to assess both attack effectiveness and performance preservation. More metrics are expected, e.g., the number of nudity parts in a generated image. FID scores or LPIPS scores to measure performance preservation for benign prompts. 

Adaptive attacks are shown in Appendix. I would say these are the most important results for such a defense work. I suggest to focus on these adaptive attacks, and show detailed results in main body using more fine-grained metrics for both attack effectiveness and performance preservation. 

Eventually, a strong attacker can remove any safety alignment with enough fine-tuning data. An extreme case is that the strong attacker re-trains a model without safety alignment given enough tuning data and computation resources. So, a right question to ask is how much more resources (data and computation) an attacker needs in order to remove the safety alignment. The paper can benefit from showing such experimental results. 

It's also helpful to show images generated for unsafe prompts (adaptive attacks) and safe prompts, for different methods.
Yes
No ethics review needed.
Would the performance of non-fine-tunable mechanism maintain its effectiveness if PATRONUS is implemented on smaller or resource-limited T2I models?
The paper's main strengths lie in its method of applying the concepts of non-fine-tunable learning and inseparable content moderation to white-box adversaries in T2I models. it demonstrates a clear understanding of model architecture, with the authors skillfully leveraging each component to optimize the defense mechanism effectively.
The paper proposes PATRONUS, a new defense framework that protects text-to-image models from white-box adversarial attacks. With a special emphasis on the threats resulting from model fine-tuning, PATRONUS represents an internal moderator and a mechanism of non fine-tunable learning rather than traditional defenses such as content moderation or alignment, which could be manipulated by attackers even at the parameter level. The internal moderator transforms unsafe input features into zero vectors, ensuring that benign features are decoded correctly while rejecting unsafe prompts. In this regard, the authors prove the efficacy of PATRONUS against different adversarial attacks with much stronger robustness compared to the state-of-the-art methods by retaining low CLIP scores on unsafe inputs while keeping performance on benign inputs.
One limitation to the method could be expensive computational resources. Using 4 A100-80GB GPUs might not be ideal for all the users who wish to use this defense method.
Yes
No ethics review needed.
1. 4.2.2 (Feature Space Calibration) mentions 'However, there is a gap between the feature distributions of the encoder output and the diffusion output, leading to PATRONUS’s occasional failure in the practical T2I working scenario.' Does this gap occur if the text input is left blank during alignment? Could you clarify the benefit of this calibration—does it primarily enhance unsafe content filtering, benign content preservation, or resistance to malicious fine-tuning? Additionally, could you also explain similar questions on VGG smoothing?

2. The paper mentions that “Φ is our simulated fine-tuning strategy set.” Could you specify the strategies within this set? Also, what impacts might arise if the defender uses different strategies from those of attackers?
1. The paper addresses a timely issue—preventing unsafe content generation—and shows effectiveness both in mitigating unsafe outputs and preserving benign ones. It also introduces a defensive measure against adversaries attempting to fintune the model toward unsafe outputs by using an adversarial training scheme.
2. The system is well-designed, with clearly defined objectives, and each loss term is crafted to meet a specific goal, making the approach easy to follow.
3. I appreciate the novel ideas, such as fine-tuning the safety filter within the decoder to act as a conditional decoder and the effective strategy of transforming the min-max optimization into min-min in Equation 23. Both are technically sound and effective solutions.
This paper presents a framework to protect text-to-image models from generating unsafe content. The framework focuses on three key goals: (1) blocking unsafe content by embedding a safety filter in the decoder, and adjusting the diffusion process to ensure unsafe outputs result in black images, (2) resisting malicious finetuning through adversarial training that simulates such funtuning, and (3) maintaining the quality of benign content by adding a term that preserves it during generation. The design clearly outlines each goal and demonstrates effectiveness through experiments.
1. The non-fine-tunable learning scheme seems largely based on Deng et al. (2024), so I question its contribution as a major novel element. However, I acknowledge that the loss function has been modified for the text-to-image task, providing some level of novelty.

2. The evaluation metrics for defending against unsafe generation only use the CLIP score, but this may not sufficiently distinguish your method from others, given the relatively small range of differences (e.g., values of 17-28 in Figure 3 for seven methods against I2P and 16-23 in Figure 4 for Sneakyprompt). Additional metrics are suggested, like NRR from SafeGen. Also, calculating the similarity between your generated images and a black image (your intended output for unsafe prompts) might provide a more direct assessment.

3. Some strategies lack clear explanations of their benefits and would benefit from experimental support. For example, Section 4.2.2 on Feature Space Calibration and the use of VGG for smoothing could be better justified. Please see detailed questions in the specific sections.

Minor issues:
1. There are inconsistencies between some figures and the descriptions, e.g., L_bd/L_fc in Figure 2 vs L_cd//Lfsc in EQ 11.
2. Some notation in the equations is hard to follow; it would help to provide explanations for each symbol directly near the equation.

I may consider raising my score if my concerns and questions are adequately addressed in the rebuttal.
@inproceedings{
anonymous2024promptindependent,
title={Prompt-Independent Safe Decoding to Restrain Unsafe Image Generation for Text-to-Image Models against White-Box Adversary},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=bzB7OIbITu},
note={under review}
}
Text-to-image (T2I) models, developed through extensive training, are capable of generating realistic images from textual inputs, profoundly influencing various facets of our lives. Nevertheless, they can be exploited by adversaries who input malicious prompts, leading to the creation of unsafe content and posing serious ethical concerns. Current defense mechanisms primarily rely on external moderation or model modification, but they are inherently fragile against white-box adversaries who have access to the model's weights and can adjust them accordingly. 

To address this issue, we propose \sys, a novel defense framework that governs both the diffusion and the decoder module of the text-to-image pipeline, enabling them to reject generating unsafe content and resist malicious fine-tuning attempts. Concretely, we first fine-tune the diffusion and the decoder module with the denial-of-service samples: 1) for the diffusion module, the inputs are unsafe image-caption pairs, the ground truth is zero predicted noise, and 2) for the decoder module, the inputs are unsafe generations from the diffusion, the ground truth is zero decoding. Then, we employ adversarial training to ensure this denial-of-service behavior for unsafe queries remains effective even after the adversary's fine-tuning with unsafe data. Specifically, we continuously simulate potential fine-tuning processes that the adversary might adopt and expose them to the model, enabling it to learn how to resist.

Extensive experiments validate that \sys effectively prevents the generation of unsafe content without compromising the model’s normal performance. Furthermore, our method demonstrates robust resistance to malicious fine-tuning by white-box adversaries, rendering it resource-intensive to corrupt our protected model, thus significantly deterring the misuse of our model for nefarious purposes.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Text-to-image generation AI security Model compliance
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/268eb0338d6ee1bd99b11b61383e589ff2036ac5.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Prompt-Independent Safe Decoding to Restrain Unsafe Image Generation for Text-to-Image Models against White-Box Adversary
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission