UIP2P: Unsupervised Instruction-Based Image Editing via Cycle Edit Consistency
Unsupervised learning;Diffusion models;Cycle edit consistency;Instruction-based image editing
Yes
No ethics review needed.
Please see the weaknesses part.
1. The motivation for removing the triplet is very meaningful and reasonable in instruction-based image editing. The bias of training data is also an important problem. 

2. The presentation is easy to understand and the discussion for the motivation is reasonable and convincing.

3. There are several experiment evaluation types including quantitative, qualitative, and user study.
The paper is motivated to remove the dependency on the triplet data of before-and-after images and editing instructions in training an image editing model and also aims to remove the bias introduced by the automatically created triplet data, which is meaningful and objective. To achieve this, the author proposes the cycle edit consistency training that does not require the edited image. Specifically, the author designs several consistency losses in various aspects such as images, pixels, and attention. The author evaluates the method on magic brush test data.
Weaknesses and Questions:

1.	Backpropagation efficiency: The author proposes several consistency losses. Considering the diffusion process InsP2P includes multiple steps to get the final image, what is the backpropagation process of the loss constructed on the image? I think this will be a recursive procedure since the loss will have to backpropagate from the last step of diffusion to the very first step which is insanely computationally expensive and requires large GPU memory.  The same questions apply to attention map consistency, etc.

2.	Validation for removing bias in InsP2P apart from inconsistency. I think the proposed method can maintain the consistency of forward and reversed edits but cannot guarantee that the edited results are good, disentangled, and do not include any bias from the training data. For example, if InsP2P edited results are not good, they can still be edited back to the original image. How does the method avoid this? Besides, which loss contributes most to the model? The table 4 does not clearly show this.

3.	The structure preservation. I think the CLIP similarity loss makes the model have the editing ability. However, this does not indicate the model can preserve the structure since the edited image can be very different from the original but have the same semantics and be edited back to the original. The author may elaborate on this.

4.	Evaluation metrics and results. To validate the editing ability, I think the author should evaluate more diverse editing types and datasets such as PIE. 

5.	I am also interested in the attention map visualization in forward and reversed editing. I think the attention can be different at the same timestep since the forward and reverse process are opposite.  Which attention map at which timestep is enforced with consistency? More clarifications and visualization evidence on the loss should be provided.

6.	Besides, the equations are not numbered.
Yes
No ethics review needed.
1. One thing I am curious about is whether the proposed method generates different editing results for the same input during inference. Is there a need to manually select the better images? Similarly, do the compared methods also select one result from multiple outcomes for comparison?
2. I noticed that the backbone structure of the proposed method is InstructPix2Pix. Therefore, during training, does this method use the pre-trained weights of InstructPix2Pix, or does it start training from scratch?
1. This paper is well written and easy to follow.
2. The task is interesting and the problem setting is reasonable.
3. Although the idea of cycle consistency has been seen in CycleGAN, proving that it can be successfully applied to instruction-based image editing also represents a certain contribution.
This paper proposes an unsupervised model for instruction-based image editing. Existing supervised methods rely on datasets consisting of input images, edited images, and editing instructions, which limits their generalization. This paper introduces Cycle Edit Consistency (CEC) to avoid this heavy dependency. Specifically, the authors apply a forward and backward edit in one training step and enforce consistency in both the image and attention space with CLIP.
Due to the lack of GroundTruth, it is difficult to make an accurate comparison of the experimental results. Looking at the edited image results, the proposed method does not seem to have a clearly superior advantage compared to the methods against which it was benchmarked. For example, in the first row of Figure 4, I prefer the result of HIVE; in the second row, I favor MagicBrush; in the third row, I like the results of MagicBrush and MGIE; in the sixth row, I am more inclined towards MGIE's result; in the eighth row, I prefer MAIE's outcome; and in the last row, I favor the results of InstructPix2Pix and SmartEdit.
Yes
No ethics review needed.
1. Are there multi-turn results contained in the paper?
2. UIP2P is trained based on instruct-pix2pix. Can UIP2P be trained from scratch (e.g. Stable Diffusion checkpoint)? Or Does this method require paired instructions pre-trained?
3. It is interesting to see that UIP2P can edit the image in fewer DDIM steps. Why? Any possible discussions?
4. In Fig.3. I am confused about clip Loss. The clip loss is added between the input caption, edited caption, input image, and edited image. Why this loss is added only between the input image and the edited image? What about the clip loss on the reconstructed image?
- Editing the images via instruction is natural. The dataset collection is expensive. This method provides an unsupervised method to achieve this goal and several loss functions are utilized to make it happen.
- The experiment results are detailed. Different datasets are trained to show the differences. Single-turn and multi-turn editing are considered.
This method presents a new method for instruct-based image editing. Different from the previous method, this method does not require paired editing images for training via cycle edit consistency. By training on the real images only, the overall results show much better performance than the previous supervised methods.
- This method involves several loss functions for training, which might make it hard to find the best checkpoint.
- There are many loss functions in UIP2P. However, the author only does the ablations on the L_sim and  L_attn. What about other loss functions?
We propose an unsupervised model for instruction-based image editing, introducing Cycle Edit Consistency (CEC) to eliminate the need for ground-truth edited images, enabling scalable and precise edits on real image datasets.
@inproceedings{
anonymous2024uipp,
title={{UIP}2P: Unsupervised Instruction-Based Image Editing via Cycle Edit Consistency},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=PNiqWDAtPq},
note={under review}
}
We propose an unsupervised model for instruction-based image editing that eliminates the need for ground-truth edited images during training. Traditional supervised approaches depend on datasets containing triplets of input image, edited image, and edit instruction, often generated by either existing editing methods or human-annotations, which introduce biases and limit their generalization ability. Our model addresses these challenges by introducing a novel editing mechanism called Cycle Edit Consistency (CEC). We propose to apply a forward and backward edit in one training step and enforce consistency in both the image and attention space. This allows us to bypass the need for ground-truth edited images and unlock training on datasets comprising either real image-caption pairs or image-caption-edit triplets. We empirically show that our unsupervised method achieves better performance across a wider range of edits with high fidelity and precision. By eliminating the need for pre-existing datasets of triplets, reducing biases associated with supervised methods, and introducing CEC, our work represents a significant advancement in unblocking scaling of instruction-based image editing.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Unsupervised learning Diffusion models Cycle edit consistency Instruction-based image editing
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/c5788a91f10e8497474807582a51537e817c5148.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
UIP2P: Unsupervised Instruction-Based Image Editing via Cycle Edit Consistency
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission