$InterLCM$: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration
diffusion model;face restoration
Yes
No ethics review needed.
All my concerns are listed in the weakness part. The main concern is about the key source of performance, which requires the author’s justification for the method design. I will consider raising my score if the author provides a thorough explanation.
(+) The proposed method is carefully designed and based on an interesting observation about LCM.

(+) The paper is well-written and easy to follow. The figures are well-designed and professionally presented, clearly illustrating the main takeaway.

(+) The experiments are thorough and solid, with especially useful discussions in the appendix.

(+) The performance is solid, outperforming baselines in terms of visual quality on real-world images across many cases.

(+) Thanks to the advantages of LCM, the inference speed is excellent, which could benefit the related community.
The authors propose using the LQ image as an intermediate state in the LCM. Because of how LCM works, they can apply image-based loss during training. The method uses the diffusion model’s prior by adding noise to the LQ image. This drags the LQ image to the domain of generative capability.  And then they guide diffusion model to restore the LQ image with constraint of semantic related info.
(-) What does the author mean by "back to the low-quality image initialization" in line 100? Does Figure 3 show all the trainable components? It seems the authors are suggesting that some latents are also trainable.

(-) Table 2 should be reorganized. The current version is squeezed and difficult to read.

(-) It seems the whole pipeline has a strong capability to preserve identity. I am curious about how the method performs when the face in the LQ image has additional textures, like tattoos or festival-style face paint. How would the proposed method handle these cases?

(-) Does the author's method design draw significant inspiration from general image restoration approaches built on diffusion models? I ask this because using a ControlNet-like design to extract useful information from low-quality images is not a new concept. Meanwhile, it seems that the main performance improvement comes from the spatial encoder, which is the ControlNet as semantic info encoder, as demonstrated in Tables 1 and 2. Additionally, the second main source of performance appears to be the LPIPS loss. It seems that one could solely rely on the spatial encoder to achieve good results. Could I understand that the performance improvements on your task could be easily obtained by simply using a ControlNet to extract LQ's semantic info? If this is not the case, please provide a justification.
Yes
N/A
No ethics review needed.
1) Could the authors provide examples of the semantic information from HR and LR images of the same scene such that one could evaluate whether the semantic information from the LR image is sufficient as a prior for HR reconstruction?
2) How were the hyperparameters (2) chosen in this study?
3) Why was a four-step LCM considered instead of a different number of steps? This choice seems heuristic.
4) Figure 1 is unclear. It says that LCM maps directly to the real image space. Does it imply LCM learns a mapping from LR to HR images directly like [2], rather than progressively denoising as depicted in Figures 2 and 3?
5) What is the degradation process considered in the synthetic dataset? Is it a simple interpolation?
1) The comparison against various baselines is comprehensive and well-organized.
2) Figure 2 and Figure 3 clearly illustrate the key concepts of the method.
3) The qualitative results demonstrate a significant improvement over several baselines.
This paper proposes a new method for blind face image restoration, consisting of three main components: 1) applying a latent consistency model (LCM) for fast sampling; 2) starting from a low-resolution (LR) image instead of standard Gaussian noise during inference; and 3) incorporating semantic information from the LR image. The authors compare their method against numerous baselines and achieve better quantitative performance over most metrics.
1) The paper overlooks important literature [1], which discusses accelerating diffusion models for inverse problems, including image restoration, by beginning with a better initialization rather than standard Gaussian noise. This prior work shares a similar concept of initiating the diffusion process from LR images in this study.
2) One motivation for employing LCM appears to be the acceleration of the diffusion prior. However, the manuscript simply states that “diffusion-prior based approaches still suffer from time-consuming inferences” (line 161) without discussing the extensive literature on accelerating diffusion models, such as techniques involving distillation (beyond the consistency model) and diffusion bridges [2].
3) The abstract is difficult to follow. I recommend revising it to make it shorter and more focused, allowing readers to easily grasp the key ideas of the paper.
4) While the quantitative metrics indicate improvements, the visual results do not convincingly demonstrate superior performance. For instance, compared to the ground truth, the proposed method generates features that differ significantly (e.g., hair in Figure 6), similar to other baselines. From a perceptual quality standpoint, it is challenging to determine whether the proposed method is indeed better. In Figure 6, the proposed method oversharpens the image compared to the reference. This concern is further supported by only marginal improvements in perceptual metrics over the baseline methods, such as CodeFormer (with a worse FID score than some baselines and a marginal improvement of 0.004 in LPIPS and 1.03 in MUSIQ).
5) Some statements in the manuscript appear overly assertive without adequate support from references or experimental evidence. For example, the abstract claims that “the latent consistency model shows more semantic consistency in the subject identity,” and line 163 states that “the commonly used perceptual loss in image restoration tasks cannot be well integrated into their framework.”
6) The contributions of this paper seem somewhat incremental. The use of LR as a prior has been investigated in [1], and LCM for fast diffusion prior is a well-established technique. From this perspective, the main contribution appears to be the incorporation of semantic information from LR images, which raises questions about whether it meets the standards for ICLR.

[1] Chung, Hyungjin, Byeongsu Sim, and Jong Chul Ye. "Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.

[2] Liu, Guan-Horng, et al. "I $^ 2$ SB: Image-to-Image Schrödinger Bridge." International Conference on Machine Learning. PMLR, 2023.
Yes
No ethics review needed.
1. The training efficiency is wondered, as the LCM backbone is freezed, how long could we get the model, and the training stability is concerned when various loss functions are added.
2. What's the difference between the spatial encoder and ControlNet, only incorporating the visual embedding?
1. This paper proposes an efficient method to utilize the generative prior for blind face restoration, that the LCM models enjoy more semantic consistency compared to the naive diffusion models, and the image-space optimization benefit more restoration-specific loss constrains.
2. The semantic consistency comparison between LCM and diffusion models are well illustrated, and the method design is straightforward without bells and whistles. The intermediate state validation and ablation experiments are reasonable and sufficient.
3. The presentation is well for readability.
This paper proposes to regard the low-quality image as the intermediate state of LCM models for blind face restoration, considering that the LCM enjoy superior semantic consistency compared to the naive diffusion models, in terms of the generative prior utilization. Experiments demonstrate the effectiveness of the proposed method.
1. Is there any comparison for x0-prediction-based diffusion model.
2. Why only face dataset, whether the proposed method could generalize to other natural image dataset, is there any discrepancy in adopting the LCM models as generative prior. If so, what modifications should we care to apply current method to other types of images, and the preliminary results on a non-face dataset would be nice if feasible.
3. Whether the proposed method can be integrated with LCM-LoRA for more fast inference? Is there any potential challenge, and the preliminary results would be recommended if feasible.
Yes
No ethics review needed.
1. Overall, I consider retraining to be a disadvantage. However, if the model demonstrates strong performance on a range of low-quality face images in real-world scenarios, it adds value. Can the authors make the trained weights available for testing?
2. How does the proposed method extend to images beyond just faces? If the study focuses solely on face restoration and is not even effective with images that include hands, I find it lacks generalizability and robustness.
1. The paper is well-written, the experiments are systematic.
2. The authors manage to outperform the prior works on blind face restoration.
The authors propose to use a latent consistency model to solve blind face restoration problems. Prior work on this research topic normally fine-tune a diffusion model on restoration datasets. To enhance the semantic consistency and incorporate the perceptual loss, authors further consider the low-quality image as the intermediate state of LCM models. The algorithm is tested on standard datasets and leads to improved performance over the considered baselines.
1. Regarding the contribution. Using pre-trained diffusion model as the image prior for blind face restoration is widely studied. The extension from conventional DDPM to LCM is quite straightforward.
2. Considering the combination of different training objectives, it is better to provide further analysis on the choices of hyper-parameters, and discuss about the risk of human bias due to these manual parameters.
3. I do not think the computation of perceptual loss is particularly challenging. Several existing works [1][2] on blind inverse problems have successfully integrated this term into a Bayesian framework.
[1] Parallel Diffusion Model of Operator and Image for Blind Inverse Problems (CVPR2023)
[2] Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution (WACV2024)
Yes
No ethics review needed.
see in weakness
1.	the authors propose a LCM-based face restoration, a simple but effective BFR framework, it effectively maintain better semantic consistency in face restorations.
2.	LCM-based method shows faster inference than stable diffusion model
The latent consistency model (LCM) demonstrates improved semantic consistency in subject identity, structural information, and color preservation, making it a viable alternative to conventional diffusion model priors. In this study, the authors propose InterLCM, which utilizes the low-quality image as an intermediate state in LCM models to facilitate efficient blind face restoration. Extensive experiments showcase the superior performance of InterLCM compared to existing approaches .
1.it is nice to find some trials on the LCM model for blind face restoration. But there exists some questions 1). Why prefer the bfr task not for the general sr task.2). the usage of lcm is straightforward, and the theory analysis and support is absent. 
3).it is better to add more relevant works using the defined priors:

1). 3D Priors-Guided Diffusion for Blind Face Restoration
2). Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild
3). Face Restoration via Plug-and-Play 3D Facial Priors
By regarding the lq image as intermediate state of the LCM, this paper propose the method InterLCM, along with extra conditions as visual embeddings and spatial embeddings, for efficient blind face restoration.
@inproceedings{
anonymous2024interlcm,
title={\$Inter{LCM}\$: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=rUxr9Ll5FQ},
note={under review}
}
With diffusion models (DMs) to powerfully estimate data distributions, diffusion priors have been explored to address the blind face restoration (BFR) problem. They normally fine-tune a diffusion model on restoration datasets to recover low-quality images. 
However, the naive application of DMs presents several key limitations. 
(i) The diffusion prior has inferior semantic consistency (e.g., ID, structure and color.),  increasing the difficulty of optimizing the BFR model.(ii) the diffusion models require hundreds of iterations in the denoising phase, as such they cannot incorporate (perceptual) losses in the image domain. These losses are crucial for BFR where faithfulness of utmost importance.As a replacement to the conventional diffusion model priors, we observe that the latent consistency model (LCM) shows more semantic consistency in the subject identity, structural information and color preservation.
Based on this observation, we consider the low-quality image as the $\textit{intermediate state of LCM models}$ for efficient blind face restoration, which we termed $\textit{InterLCM}$ . We found that starting from the early step of LCM is helpful to achieve the trade-off between the fidelity and quality of the generated images.
Since the LCM model maps the noisy latent space to real data during each time step, we can also successfully integrate perceptual loss during training of \ourmethod, an approach not yet explored by existing methods, leading to substantial improvements in image quality, especially when applied in real-world scenarios.
However, the straightforward application of the few-step diffusion model introduces uncertainty to the structure and semantics of the restored image. 
In our method $\textit{InterLCM}$ , the low-quality image is further imported into a Visual Module to extract visual features and a Spatial Encoder to extract spatial information to enhance fidelity.
Extensive experiments demonstrate that $\textit{InterLCM}$ outperforms existing approaches in both synthetic and real-world datasets while also achieving faster inference speed. The source code and models will be public.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
diffusion model face restoration
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/954b3df0cd3e76fcdc139e0f45c58d2df1ed0300.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/6eddc1c8eb6bd9531b3453e3c54283a3b23dcbb1.zip
$InterLCM$: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission