TREANT: Red-teaming Text-to-Image Models with Tree-based Semantic Transformations
Red-teaming
Yes
No ethics review needed.
Check weakness section
1. The proposed method is able to adjust and refine prompts using semantic decomposition and sensitive element drowning enhances its practical utility and effectiveness.
2. The experimental results demonstrate the efficacy of TREANT through comprehensive evaluations, achieving an overall success rate of 88.5% on leading T2I models like DALL-E 3 and Stable Diffusion.
This paper proposed red-teaming framework for adversarial testing of T2I models to detect their ability to generate NSFW content. Specially, the proposed TREANT utilizes  tree-based semantic transformations to systematically refine adversarial prompts for effective testing.
1. Missing related work white-box adversarial prompt attack baselines such UnlearnDiffAtk [1], which is even able to enforce unlearned T2I model to generate images containing harmful contents. 

2. To demonstrate scalability, it is better to add time efficiency as one main evaluation metric.

[1] To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images … For Now, ECCV’24
Yes
No ethics review needed.
See the weakness for the questions.
The TREANT framework has several strengths that make it effective for testing text-to-image (T2I) models. It achieves a high success rate of 88.5% in generating unwanted content, demonstrating its reliability. 

TREANT operates automatically, allowing for quick and easy testing without manual effort. It uses smart techniques to bypass safety checks on both text and images while maintaining the original meaning of prompts. The organized tree structure helps manage prompts efficiently, reducing wasted queries. 

Additionally, it has been thoroughly tested across various scenarios, proving its robustness, and the authors provide open access to their code and data for others to use.
I think this paper is targeting at an important problem and the method seems to be very effective. More specifically, this paper presents TREANT, a tool for testing text-to-image (T2I) models to see if they can produce inappropriate content. TREANT works by changing sensitive words in prompts to less sensitive ones and adding harmless details to help avoid safety checks. This approach has a high success rate of 88.5% in getting T2I models to generate unwanted images, performing better than other methods. The tool aims to improve how we evaluate the safety of these models.
1. It would be great if the authors could include more qualitative results in their paper because it would really help us understand how well TREANT performs. The success rate of 88.5% sounds impressive, but sharing some specific examples of the images generated would give us a clearer idea of what’s going on. It would be useful to see the types of content TREANT produces and how it handles different situations. I get that safety concerns might limit what they can show, but if they can share more examples, it would definitely make the paper more engaging and helpful for readers. E.g. in figure 7 it would be better if more examples can be shown for a more convincing conclusion,

2. And more analysis together with the qualitative results could be more helpful.

3. Besides, while TREANT is trying to improve scalability, it would be really helpful to talk more about how well it can work with different models and handle prompts that are more complex than the ones they tested.
Yes
This paper focuses on red-teaming T2I models for NSFW content generation. The paper contains some NSFW examples and may cause harmful effects.
Yes, Potentially harmful insights, methodologies and applications Yes, Responsible research practice (e.g., human subjects, data release)
1. For red-teaming, a white-box attack should also be acceptable. Is there any special case for the need for black-box attacks?
1. Red-teaming T2I models is an important topic to investigate. It is also a practical issue.
2. Overall, this paper is easy to follow and well-organized.
3. The proposed method is easy to implement and can perform black-box red-teaming, which should be efficient.
The paper introduces TREANT, a novel automated red-teaming framework designed to enhance adversarial testing of T2I models. It utilizes tree-based semantic transformations, semantic decomposition, and sensitive element drowning strategies, leveraging LLLMs to refine adversarial prompts. TREANT demonstrates superior performance in generating NSFW content on DALL·E 3 and Stable Diffusion.
For method:

1. The proposed method lies in the category of prompt engineering, unaware of the T2I model and without any black-box optimization strategy. I fail to see clear novelty compared to token replacement and insertion. Especially, in some cases, this method performs worse than BAE or even SneakyPrompt.
2. It would be better if the authors provided any form of guarantee or evidence showing the proposed algorithm can cover more possible prompts toward NSFW semantics.
3. It is misleading that the proposed method is designed to bypass image safety filters. However, GPT-4o is used to check whether NSFW content is generated, which means GPT-4o can serve as a good image safety filter to drop all the malicious prompts generated by TREANT.

For evaluation and comparison:

4. The paper fails to include important baselines, such as [a].
5. Only unprotected models are considered, the authors are encouraged to investigate the effectiveness of models such as SLD[b].
6. Please try to include DiT-based models like PixArt-α[c].
7. Please consider more image safety filters for evaluation, and also check the attack transferability.

[a] MMA-Diffusion: MultiModal Attack on Diffusion Models. Yang et al. CVPR 2024.

[b] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. Schramowski et al. CVPR 2023.

[c] PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis. Chen et al. ICLR 2024.
Yes
No ethics review needed.
- Clarification on what you mean by "misleading the safety filter" as opposed to "bypassing" it? A clear distinction would be helpful, since you state that your method generated adversarial prompts that actually bypass the safety filters of T2I models.
- How do you convert the PPT back into a prompt (your final adversarial prompt) after decomposition? Is this done manually or through an automated process, such as querying a LLM?
- Provide further clarification on how the method determines whether the adversarial prompt is blocked by the text filter or the image filter, as different red-teaming strategies are triggered in your method.
- Your method seems to rely heavily on querying a LLM for constructing the PPT, performing semantic decomposition, and implementing sensitive drowning. More details on how you ensure the LLM accurately understands the task, particularly in terms of constructing the PPT correctly?
- Since models like SD1.4, SD2.1, and SDXL have weaker safety mechanisms compared to DALL·E 3, it would be helpful to provide more explanation on why the results for these models are not better than the baselines. The mention of these T2I models potentially not understanding the structural information of the language need more elaboration. Is your generated adversarial prompts somewhat different from your baselines?
- In Table 1, the success rate of TREANT-SED appears quite low. It might be useful to discuss this in more detail, particularly since TREANT-SED employs prompt dilution (adding non-sensitive information into the prompt), which has been shown to be effective in many prior studies.
- The paper provides a clear and understandable explanation of how T2I model safety mechanisms work, especially focusing on DALL·E 3, which is widely regarded as difficult to jailbreak.
- The high success rate in jailbreaking DALL·E 3 is impressive, showcasing the method's robustness and effectiveness.
- The introduction of the Prompt Parse Tree (PPT) and the recursive editing process to construct adversarial prompts is a novel and interesting approach.
- The method may prove to be query-efficient, which could be beneficial for practical applications.
This paper introduces an automated black-box red-teaming framework designed for adversarial testing of T2I models. The proposed method leverages a tree-based semantic transformation approach to recursively refine prompts, aiming to bypass both text and image safety filters. Two key red-teaming strategies semantic decomposition (probe text safety filter) and sensitive element drowning (probe image safety filter) are applied to transform sensitive prompts into adversarial ones.
- The method of semantic decomposition could benefit from further clarification (e.g., in line 253 you say you query the T2I model, but in your algorithm you query LLM, so which do you query). It would be helpful to provide more details on how this process works.
- The construction of the Prompt Parse Tree (PPT) is interesting, but it’s not fully clear why it’s beneficial for the method. Could strategies like semantic decomposition and sensitive drowning be performed without the need for PPT? Explaining why PPT is essential would strengthen the argument.
- The comparison baselines in the experiment section could be expanded. Testing against more robust safety mechanisms, such as those in Stable Diffusion variants like ESD [R1], SLD [R2], and others, would provide a more convincing case for the method’s effectiveness.
- The method seems to be tailored specifically to DALL·E 3, which raises questions about its generalizability to other T2I models. More evaluation on different models could be valuable.

[R1] Gandikota, Rohit, et al. "Erasing concepts from diffusion models." ICCV (2023) \
[R2] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models." CVPR (2023)
We present TREANT, an automated framework that enhances adversarial testing of text-to-image models like DALL·E 3 and Stable Diffusion, achieving 88.5% success in generating NSFW content.
@inproceedings{
anonymous2024treant,
title={{TREANT}: Red-teaming Text-to-Image Models with Tree-based Semantic Transformations},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=PTgTlj6x0W},
note={under review}
}
The increasing prevalence of text-to-image (T2I) models makes their safety a critical concern. Adversarial testing techniques have been developed to probe whether such models can be prompted to produce Not-Safe-For-Work (NSFW) content. Despite these efforts, current solutions face several challenges, such as low success rates, inefficiency, and lack of semantic understanding. To address these issues, we introduce TREANT, a novel automated red-teaming framework for adversarial testing of T2I models. The core of our framework is the tree-based semantic transformation. We employ semantic decomposition and sensitive element drowning strategies in conjunction with Large Language Models (LLMs) to systematically refine adversarial prompts for effective testing. Our comprehensive evaluation confirms the efficacy of TREANT, which not only exceeds the performance of state-of-the-art approaches but also achieves an overall success rate of 88.5% on leading T2I models, including DALL·E 3 and Stable Diffusion.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Red-teaming
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/2894c1479e5cf318c73391cd0891c90f7802f34f.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
TREANT: Red-teaming Text-to-Image Models with Tree-based Semantic Transformations
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission