Conditional Diffusion on Web-Scale Image Pairs leads to Diverse Image Variations
diffusion;image variations;generative models
Yes
No ethics review needed.
Refer to Weakness.
1. This paper is easy to understand.
2. The proposed method achieves good results on several benchmarks.
This paper focuses on the task of generating image variations and proposes the Semantica model. Unlike traditional diffusion model training, Semantica is trained on web-scale image pairs. It receives a random (encoded) image from a webpage as conditional input and denoises another noisy random image from the same webpage. Additionally, the paper explores the impact of different image encoders on the results.
1. I have some doubts about the task of generating image variations. The model trained in this paper only supports image input and does not support text guidance, so it doesn’t seem to have strong practical value for real-world image editing. Additionally, many AIGC models already support various tasks such as image editing and style transfer. Therefore, I don't quite understand the value of the generating image variations task.
2. Emu2 [1] has already discovered that using CLIP image embeddings as conditional inputs for training a diffusion model can successfully reconstruct images, which is similar to the findings in this paper.
3. From the visual results, it is difficult to see any advantage of the proposed method over IP-Adapter. Moreover, IP-Adapter has broader application scenarios, as it can adapt to various text-to-image models.



[1] Generative Multimodal Models are In-Context Learners. CVPR, 2024.
Yes
No ethics review needed.
1. In eqn 1, why there is 2 variable t? One of them is context?
2. In eqn 2, conditioning input is $y_c$, but target is x? which seems different with L301-303
* Generating diverse image variations by training on images from the same webpage is both simple and interesting.
* The paper is well-written and easy to follow. The visualizations seem to outperform the prior works listed in the paper.
* The designs and ablation studies of different model components are well explained.
This paper introduces a new model, Semantica, which generates image variations given contextual input images. The model is trained on web-scale image pairs. Specifically, it is trained with pairs of images from the same webpage. Multiple design choices are discussed in the paper, such as the selection of the image encoder, diffusion decoder, etc. The model is evaluated on a few-shot version of FID, recall, and precision for the generation of image variations and outperforms prior works such as SD-v2, IP-adapter, and Versatile Diffusion.
* There is a missing comparison with some related work, such as RIVAL [1] which also targets the image variation task.
* Few-shot FID, precision, and recall may not be sufficient to fully quantify performance. Some metrics proposed in the table 1 of RIVAL [1] could provide more insightful for evaluations.
* There is a lack of a user study to compare different methods, which would be more convincing, as numerical metrics may not fully quantify performance.


**Reference**

[1] Real-World Image Variation by Aligning Diffusion Inversion Chain, NeurIPS 23
Yes
No ethics review needed.
1. Image representation learning, e.g. self-supervised, image reconstruction based, contrastive learning and etc, are wildly studied. How the other encoders perform compared with DINO? e.g. MAE (ref 1)
2. Web-scale image is indeed Episodic WebLI in this paper. Are there any other web-scale images? Why choose Episodic WebLI in particular?
3. Please explain robustness of the method with respect to the cleanest of the web-scale images (line 306-315), e.g. what if the web-scale images are not well filtered out, how the method perform then?
4. As explained in the paper, the commonly used metrics, e.g FID, fail to evaluate variation performance. However, it's still not clear how the proposed solution (sec 7) can be effective in measure diversity, e.g. Fig. 3 should be further analysed to explain diversity superiority.

re1: Masked Autoencoders Are Scalable Vision Learners, CVPR 2022
1. With web-scale images, this paper successfully find a good application on top of it to perform Image variation generation, which is inspiring and effective as illustrated in the experiments section.
2. The final conclusion that DINOV2 produces image representation for better image variations is also interesting, explaining self-supervised representation learning is promising in achieving effective image generation/editing.
3. The proposed new way of evaluate performance of image variation techniques seems interesting and inspiring.
This paper works on image variations generation based on two main observations. Firstly, conditional diffusion models with frozen condition embeddings can reconstruct the image with minor variations. Secondly, web-scale image can be used to train image variation models. This paper then examine various design choices for condition representation generation given its crucial role in the proposed method, and find DINOV2 can provide good image representation for better image variation generation.
1. The analysis on image encoder part for image variation (sec 2.2) seems not informative enough, although it's one main contribution of the paper, as it's common sense that the effectiveness of conditional representation is crucial for image generation.
2. A new application of web-scale image for image variation is not well explained. It's not clear how the authors come up with this idea of using web-scale image for image variation, making it hard to evaluate the significant of the contrition.
3. Data filtering should be clearly introduced as it's one of the main contribution. However, Sec. 5 is not clear (see Questions 3).
4. The experimental results section seems weak in proving superiority of the proposed solution. (see Questions 4)
Yes
No ethics review needed.
Please provide specific examples or quantitative evidence demonstrating how LPIPS and FID fail to capture diversity in image variations and should do a comparative analysis between the proposed one-shot metrics and LPIPS/FID, including specific examples where the new metrics better capture diversity.
- The idea of using the semantic relations of image pairs with in the the same web page is a good contribution which can lead to even more  research direction then the focus of this paper
- The observation of image diversity issue in current SOTA , mitigate it and suggest new metric to evaluate it
The paper if focus on the challenge of generating image variations, with large diversity while preserving the the conditions' semantics.
The authors suggest a new pre-training method exploiting the semantic relations of images within the same web page.   
It also discuss the challenge of measuring image diversity and suggest a few-short metrics approach
The claim that "Standard image-level metrics such as LPIPS and distribution-level metrics such as FID fail
to capture diversity in image variations" is not backup with number and/or examples
@inproceedings{
anonymous2024conditional,
title={Conditional Diffusion on Web-Scale Image Pairs leads to Diverse Image Variations},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=s7vwXDsVYa},
note={under review}
}
Generating image variations, where a model produces variations of an input image while preserving the semantic context has gained increasing attention. Current image variation techniques involve adapting a text-to-image model to reconstruct an input image conditioned on the same image. We first demonstrate that a diffusion model trained to reconstruct an input image from frozen embeddings can reconstruct the image with minor variations. Second, inspired by how text-to-image models learn from web-scale text-image pairs, we explore a new pretraining strategy to generate image variations using a large collection of image pairs. Our diffusion model \textit{Semantica} receives a random (encoded) image from a webpage as conditional input and denoises another noisy random image from the same webpage. We carefully examine various design choices for the image encoder, given its crucial role in extracting relevant context from the input image. Once trained, \textit{Semantica} can adaptively generate new images from a dataset by simply using images from that dataset as input. Finally, we identify limitations in standard image consistency metrics for evaluating image variations and propose alternative metrics based on few-shot generation.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
diffusion image variations generative models
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/2f76c32c44133acaf93014512501ef825bba42cd.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Conditional Diffusion on Web-Scale Image Pairs leads to Diverse Image Variations
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission