Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model
Face image restoration;diffusion model
Yes
Yes, Responsible research practice (e.g., human subjects, data release)
1. The design of the Dual control Adapter is a little but tricky. Can one just replace it by a transformer?
    2. There can be different  result for every sample. How diverse is the model? And how to decide which sample to use?
    3. Will be dataset be realised?
1. The motivation that using multi-modality input to assist face restoration is nice. As shown in the paper, there could be ambiguity during restoration, and the additional input could be helpful. 
2 .  The use of diffusion prior for this work also make sense, and it models a probability given the input. 
3. The dataset could be beneficial to the community.
The paper propose to utilize diffusion prior, as long as multi-modality input to perform real world face restoration. The method can receive text or reference image for face restoration. 

They also present a dataset that contains identity-image pair. 

There is both quantitative  and qualitative metric, which demonstrate the effectiveness of this method.
1. The method used in the paper is not new. The main contribution of this works seems to be on the dataset and using expositing method on a newer task. 
2. Ablation of the effectiveness of using  trained diffusion prior is missing.
Yes
No ethics review needed.
1. Not super important, but how do the authors feel the proposed method can be extended for real-world video face restoration? Can they please add some discussion on the scalability of the proposed method?
2. Overall the paper looks promising and makes meaningful contributions, however, it lacks some important experiments and details. The authors can refer to the weaknesses section.
1. The topic of real-world face restoration is interesting. 
2. The overall writing of the manuscript is clear and easy to follow.
3. This work introduces the Reface-HQ dataset, which comprises over 23,000 high-resolution images. This dataset can provide a foundation for training and evaluating the model.
This work addresses the challenge of real-world face restoration. The authors introduce the Multi-modal Guided Real-World Face Restoration (MGFR) method, which improves facial image restoration from low-quality inputs using attribute prompts and reference images. Specifically, MGFR employs a dual-control adapter and a two-stage training strategy, leveraging a dataset of over 23,000 high-resolution images to enhance visual quality, identity preservation, and attribute correction. Experimental results demonstrate that the proposed method achieves superior performance.
1. The proposed method appears complex, and its running time and memory consumption are not superior to existing methods like DiffBIR and DR2. Since running time and memory usage are crucial for real-world applications, this raises concerns about its practical feasibility.
2. There is a lack of comparison in FLOPs and Params in the main comparison and ablation study.
3. Lacking comparison to recent works (SUPIR, BFRffusion) in Table 1. 
a. Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild
b. Towards real-world blind face restoration with generative diffusion prior.
Yes
human face images are collected for training the proposed model.
Yes, Responsible research practice (e.g., human subjects, data release)
1) How will the performance vary if different versions of diffusion models are used? That is, would diffusion models be the strong foundation of achieving the good performance?

2) Some figures, say fig.4 and 5, are with too small fonts and unclear color denotions, making it less readable.
* Novel framework:

The MGFR approach introduces a novel dual-control adapter and two-stage training strategy, effectively combining multi-modal priors for enhanced face restoration. The use of attribute text prompts alongside high-quality reference images and identity information is a significant advancement, offering more control over the restoration process.

* Dataset Contribution:  
The introduction of the Reface-HQ dataset addresses a critical gap in the availability of high-resolution reference images, providing a valuable resource for the community.

* Good Performance: 
The method achieves superior visual quality in restoring facial details under severe degradation, demonstrating its practical applicability in real-world scenarios. The ability to control restoration through textual prompts enhances the flexibility and precision of the restoration process.

* Mitigation of False Illusions: MGFR effectively addresses the problem of false facial attributes and identities, a common issue in existing generative face restoration methods.
The paper introduces a Multi-modal Guided Real-World Face Restoration (MGFR) approach, which aims to enhance facial image restoration from low-quality inputs by leveraging multi-modal prior information. This includes attribute text prompts, high-quality reference images, and identity information to mitigate false facial attributes and identities typically generated by current restoration methods. MGFR employs a dual-control adapter and a two-stage training strategy to effectively utilize these priors for targeted restoration tasks. The paper also introduces the Reface-HQ dataset, containing over 23K high-resolution facial images from 5K identities, to support the training of reference-based restoration models. The proposed approach demonstrates superior performance in restoring facial details, particularly under severe degradation, while ensuring identity preservation and attribute correction.
* Complexity of Implementation:
The integration of multiple modalities and the dual-control adapter may introduce complexity in implementation. Therefore, I recommend that this paper should consider a more detailed information on computational requirements and scalability would be beneficial.

* Evaluation Metrics:

While the paper demonstrates superior performance, a more thorough discussion of the evaluation metrics used to assess visual quality, identity preservation, and attribute correction would strengthen the claims.

* Ablation Studies:
The paper would benefit from additional ablation studies to isolate the impact of each component (e.g., attribute prompts, reference images) on the overall performance, providing deeper insights into the effectiveness of each modality.
@inproceedings{
anonymous2024overcoming,
title={Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=m9RNBZewW2},
note={under review}
}
We introduce a novel Multi-modal Guided Real-World Face Restoration (MGFR) technique designed to improve the quality of facial image restoration from low-quality inputs. Leveraging a blend of attribute text prompts, high-quality reference images, and identity information, MGFR can mitigate the generation of false facial attributes and identities often associated with generative face restoration methods. By incorporating a dual-control adapter and a two-stage training strategy, our method effectively utilizes multi-modal prior information for targeted restoration tasks. We also present the Reface-HQ dataset, comprising over 23,000 high-resolution facial images across 5,000 identities, to address the need for reference face training images. Our approach achieves superior visual quality in restoring facial details under severe degradation and allows for controlled restoration processes, enhancing the accuracy of identity preservation and attribute correction. Including negative quality samples and attribute prompts in the training further refines the model's ability to generate detailed and perceptually accurate images.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Face image restoration diffusion model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ea9fbeeae86d60eda821bd7088d723ac65178d7b.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission