SAT-LDM: Provably Generalizable Image Watermarking for Latent Diffusion Models with Self-Augmented Training
image generation;watermarking;latent diffusion model
Yes
No ethics review needed.
Please refer to the weaknesses
- The writing is clear and easy to follow.
- The method does not require additional training data.
- A theoretical guarantee is provided for the proposed method
- Experiments show that the method produces effective watermarks while maintaining high image fidelity.
This paper studies how to watermark diffusion models across diverse image styles. The authors propose a training-based watermark method SAT-LDM. In particular, the authors plug a message processor into the VAE decoder to obtain watermarked images from latents. During the training, SAT-LDM jointly trains the message processor and message extractor. The diffusion model is fixed during the training. No external data is required for the training. Theoretical analysis is provided to demonstrate the generalization ability of the proposed method. Experiments show that SAT-LDM can generalize across different image styles.
- More visualization results, especially of different image styles, can help demonstrate the generalization of the proposed SAT-LDM.
Yes
No ethics review needed.
- What dataset did the authors use to train the message processor and extractor? How does this align with the claim of “no external data”?
- Given the potential for the message extractor to output messages from non-watermarked images, why is there no report on FPR?
- Does Figure 2 accurately represent the proposed structure? If so, how does a frozen VAE decoder process message embeddings, and what are the implications of only training the message processor and extractor?
The paper focuses on a significant topic—embedding watermarks in images generated by diffusion models like Stable Diffusion—to address the growing need for copyright protection of AI-generated content.
This paper introduces SAT-LDM, a watermarking method integrated within latent diffusion models (LDMs) to generate watermarked images. The authors argue that SAT-LDM improves generalization across image styles without compromising quality, unlike existing methods which reportedly degrade image content.
- The paper's **motivation lacks clarity and appears misguided.**    
The authors claim that current diffusion-native watermarking methods degrade image quality by comparing watermarked images to unwatermarked counterparts generated from the same prompt. This comparison is irrelevant because users of diffusion-native watermarking methods would not see unwatermarked images beforehand. Such comparisons would only apply to post-watermarking methods, where preserving the original image content is necessary.    
Additionally, the claim that watermarking performance is compromised across image styles is questionable, as style diversity is largely dictated by the diffusion model's training data rather than the watermarking process itself.

- The paper’s Figure 2 depicts a VAE decoder as frozen while taking message embedding as input. If the authors follow the FSW structure, as stated in Section 4.2, **the figure is incorrect because FSW fuses message embeddings into the UNet-decoder, not the VAE decoder**. If the figure is correct, freezing the entire diffusion model (including the VAE) while training only the message processor and extractor introduces the risk of extracting false-positive messages from non-watermarked images (i.e., images generated without embedding). Thus, **the paper should include a false positive rate (FPR) analysis to evaluate this aspect**.

- The authors claim “no external data” usage in Figure 1, but they must have used training data for the two message components, just like the existing methods.

- The approach lacks innovation, as it mainly replicates FSW’s structure and uses the spatial transformer network from StegaStamp to improve robustness.

-  The experimental evaluation lacks critical ablations and comparisons. The paper should investigate the impact of using different pre-trained diffusion models and evaluate the method under strong attacks, such as diffusion-based attacks gauge robustness under adversarial conditions. WAVES [1] could be a reference in this case.

[1] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.
Yes
No ethics review needed.
I am willing to increase my score if authors provide a detailed explanation on the assumption of Lipschitz continuity of the loss function. How to verify an assumption? Does it hold in practice?
Authors provide a theoretical guarantee on the generalization error of their approach that, to my knowledge, has not been done in the previous works. Experimental evaluation demonstrates that the proposed method yields watermarked images of better quality than of the competitor's works. The robustness to removal attacks is on the level of state-of-the-art methods.
The paper proposes an image watermarking scheme that generalizes across different image styles. The authors prove the generalization bound for the watermarked image generator and the message extractor and compare their method against several state-of-the-art approaches in terms of image quality and robustness to removal attacks. 

Please use this review sparingly.
The assumption on the Lipschitz continuity of the loss function is somewhat strong: 1) how can one check if it is true and 2) estimate the Lipschitz constant K (what can be quite large leading to unsatisfactory large upper bound)
Yes
No ethics review needed.
The questions are connected to the points above, respectively:

1. Can the authors provide results using LAION-400M prompts during testing to ensure the performance improvement is more general?
2. Can the authors offer further insights regarding Fig. 3, such as whether there exists a good subset of LAION-400M prompts that generate images similar to those from GPT-generated prompts? From Fig. 3, there appears to be significant overlap between the External and Test regions. If so, this suggests the proposed testing scenarios may be biased, given that the external training set prompts are more diverse.
3. Could the authors provide additional results and discussion on guidance?
4. Could there be any ablation study, as previously mentioned, to differentiate between the contributions of the structural and training data improvements?
1. The observation that there is a mismatch between the training and testing phases is crucial and can greatly inform future research.
2. The theoretical section is informative.
The paper observed a mismatch in the target image distribution to be watermarked in diffusion models with previous methods, i.e., external dataset (training phase) vs. generated dataset (testing phase). To address this, the authors propose improving the process by training the watermark using data generated by the diffusion model itself. This enhancement leads to improved performance in the authors' experimental settings.
Below are the main concerns:

1. The biggest concern stems from the experimental setting. The authors claim that under 1,000 AI-generated prompts, the proposed method surpasses the baseline. However, this number of prompts is far too small for general case measurements, especially considering potential bias in the language models used. Additionally, this comparison seems unfair. The authors should provide a comparison of results using selected prompts from LAION-400M (but not used during training). The reported advantage may solely come from the prompt distribution shift between GPT-generated prompts and those from LAION-400M. Since the proposed method forgoes all prompt information during training (“using empty prompt”), it avoids this shift and may appear better (due to bias from GPT-generated prompts).
2. Another crucial drawback lies in the explanation of why the method works. The authors use generated data with an empty prompt and claim that this somehow represents the conditional distribution of diffusion models, showcasing a demo experiment in Sec. 5.3 Training Distributions. However, this demo experiment needs further verification. Fig. 3 can be interpreted as $d(z_{\text{laion-prompt}}, z_{\text{gpt-prompt}}) > d(z_{\text{no-prompt}}, z_{\text{gpt-prompt}})$, which does not provide any evidence that $d(z_{\text{real-prompt}}, z_{\text{no-prompt}})$ is small in most cases. In fact, it is widely believed that there is a large distance between the conditional and unconditional distributions in DMs, which is why we use (either classifier or classifier-free) guidance.
3. Another critical point is the "guidance" aspect. The authors need to show experimental results with guidance scale = 1, i.e., direct conditional generation (without incorporating an unconditional DM). I am concerned that the real mechanism behind the method's success is that the watermarking modules adapt to the negative output of the unconditional generation. Since in all cases with guidance scale > 1, the final output probability is inversely proportional to the unconditional distribution, this might be the actual reason for the results, rather than the claim that the method accurately reflects the generated distribution of diffusion models.
4. The paper claims two improvements: training data is shifted from a dataset to generated data, and the model structure is updated in Sec. 4.2. However, it is unclear which of these contributes to the observed improvements without an ablation study. There is a risk that the gains come from the structural update rather than the proposed training data change.

Here are some additional minor drawbacks, though they do not significantly impact my overall assessment of the paper:

1. While the intuition—using generated distribution instead of training data—is clear, the writing makes this idea more complicated than necessary. While formal theory is important, the explanation could be simplified to better emphasize the intuition, with theory introduced later.
2. In LaTeX, it would be better to use ``" for double quotes instead of "".

In summary, the method might work (under the specific experiment setting) only due to the fact that the prompt distance between the zero-prompt and GPT-generated ones is closer than the distance between LAION-400M prompts and GPT-generated ones. Given that the results are measured under GPT-generated prompts, it remains unclear how promising the method truly is. Additionally, the explanation for why the proposed method works based on the DM mechanism is insufficient.

Admittedly, the idea of using the generated distribution of DMs for watermarking is potentially promising. However, using the unconditional generated distribution seems less meaningful. It might be better to use conditional generated distributions with diverse prompts, such as those from LAION-5B.
@inproceedings{
anonymous2024satldm,
title={{SAT}-{LDM}: Provably Generalizable Image Watermarking for Latent Diffusion Models with Self-Augmented Training},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=ETFfXGM3e4},
note={under review}
}
The proliferation of AI-generated images necessitates effective watermarking to protect intellectual property and identify fake content. While existing training-based watermarking methods show promise, they often struggle with generalization across diverse image styles and tend to produce noticeable artifacts. To this end, we introduce a provably generalizable image watermarking method for Latent Diffusion Models with Self-Augmented Training (SAT-LDM), which aligns the training and testing phases by a free generation distribution to bolster the watermarking module’s generalization capabilities. We theoretically consolidate our method by proving that the free generation distribution contributes to its tight generalization bound without the need to collect new data. Extensive experimental results demonstrate that SAT-LDM achieves robust watermarking while significantly improving the quality of watermarked images across diverse styles. Furthermore, we conduct experimental analyses to demonstrate the strong generalization abilities of SAT-LDM. We hope our method offers a practical and convenient solution for securing high-fidelity AI-generated content.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
image generation watermarking latent diffusion model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/dfef6f1c4ff18fa25a9bcefe0b5b94fc432ae9d5.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
SAT-LDM: Provably Generalizable Image Watermarking for Latent Diffusion Models with Self-Augmented Training
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission