Preserving the Unique Heritage of Chinese Ancient Architecture in Diffusion Models with Text and Image Integration
Diffusion;RAG;Representation Learning;Chinese Ancient Architecture
Yes
No ethics review needed.
- The paper should be revised to correct English writing mistakes and typos.
- What is the complexity of the proposed method when the authors combine many techniques?
- The research has potentials in architectural heritage preservation.
- Use of RAG and GraphRAG for enhancing text prompts with cultural background improves alignment between image content and historical context.
The paper introduces a pipeline for generating images of Chinese ancient architecture by preserving cultural and structural details in generated images. By combining many existing techniques, the proposed method can capture architectural elements in fine detail. The authors compare the proposed method with pre-trained text-to-image models, showing the effectiveness of the proposed method in image-text alignment and preservation of unique architectural traits.
- The novelty is limited, which is not enough for ICLR.
- Contributions are unclear. The proposed method is just a combination of existing techniques, which is not sufficient for ICLR.
- The paper writing is not good and need to be revised.
- The comparison is unfair. The proposed method is finetuned while compared methods are based on pre-trained models.
Yes
No ethics review needed.
What if I want to generate Chinese ancient architecture with other specific styles accurately?
1)	The topic is interesting and meaningful. The author tries to fine-tune diffusion models to work on specialized domain, such as Chinese ancient architecture. 
2)	The author introduces several models, such as SAM2 and BLIP3, to help improve the quality of image-caption pairs, so as to fine-tune the diffusion model to generate accurate outputs. 
3)	The experimental results show effectiveness of the method that can reduce the FID of Chinese ancient architecture images generated by diffusion models.
This paper aims to use diffusion models to generate Chinese ancient architecture with specific features or styles. The author uses LoRA to fine-tune a pretrained diffusion model, and proposes "Image-Annotation-Augmented Diffusion" pipeline that combines image segmentation, annotations enhanced by human feedback, and domain-specific cultural data. The approach involves using SAM2 for precise image segmentation, BLIP3 for text annotations, and GraphRAG modules to retrieve and augment cultural data. The experimental results show effectiveness of fine-tuning methods on diffusion models.
1)	The representation is poor. There are many spelling mistakes and grammatical mistakes in the paper, such as “mothods”, “catogory”, “... to proof the advantage” and “The generated results exhibits ...”. Therefore, I question the completeness of the paper. 
2)	The effectiveness of some experimental results is not obvious. For example, Fig. 5 shows the comparison of the method with other models. However, there is little difference between the outputs of fine-tuned SDXL and the paper, and the output image of paper’s method with LoRA in line 5 is not reasonable. Also, the style of reference pictures is too monotonous, which is not enough to prove the effectiveness of the method in this paper to achieve the goal of generating buildings with various specific styles. 
3)	The lack of novelty. I appreciate that the author uses SAM and other models for processing images and annotation to obtain better data quality. However, there is not enough contribution to innovation methods.
Yes
No ethics review needed.
1. In Figure 5, the generated image of Ours (LoRA) seems to depict an incorrect view for the prompt“A top-down view of palace.”

2. In Figure 1, unique feathers --> unique features?
1. This paper focuses on fine-tuning a diffusion model for synthesizing images of Chinese ancient architecture, which is beneficial for the digital preservation of historical relics and the advancement of related industries.

2. The pipeline is well-structured and practical. It considers both the subject and style of the architecture during model training. Additionally, combining RAG methods and LLM with human feedback improves the accuracy of the text descriptions.
This paper introduces an Image-Annotation-Augmented Diffusion pipeline that preserves the unique heritage of Chinese ancient architecture in diffusion models. The process begins by using SAM2 to accurately crop architectural objects from images, followed by generating text descriptions using the image captioning model BLIP3. Next, RAG, GraphRAG, and LLM models process professional texts sourced from the Cultural Relics Bureau to identify relationships between architectural elements. Finally, human feedback is applied to refine the text, producing the final text prompt for the diffusion model.
1. This paper resembles an engineering project that applies existing methods and pipelines for generating images of Chinese ancient architecture. Regarding methodological novelty, the combination of these existing methods is not particularly innovative. However, the proposed pipeline effectively addresses the unique features of Chinese ancient architecture and successfully achieves accurate image generation.

2. The experimental testing could be strengthened. For instance, attributes like dynasty, cultural style, and location are valuable for characterizing architectural features. During inference, the authors could vary one or more of these attributes to observe differences in the generated architecture. Such testing would help verify whether the learned diffusion model accurately understands key architectural information.
Yes
No ethics review needed.
1. Why not attempt to apply the proposed pipeline to other domains to further verify its effectiveness?
2. Are there any potential limitations or challenges that might arise when applying this pipeline to other types of cultural or architectural heritages?
3. How might the performance of the pipeline be affected by different levels of data availability and quality in other domains?
1. The writing is clear and understandable, facilitating comprehension of the research content and methods.

2. The experimental results appear promising, indicating the potential of the proposed pipeline in handling the specific task of generating images related to ancient Chinese architecture.
In this paper, the authors present an Image-Annotation-Augmented Diffusion pipeline integrated with human feedback. This pipeline aims to explore the specific-area paradigm for image generation considering small data amounts and professional concepts. Initially, the Segment Anything 2 (SAM2) model is utilized to acquire refined content images, enabling an in-depth examination of the relationship between unique characteristics and multimodal image generation models. Subsequently, the Retrieval-Augmented Generation (RAG) and BLIP3 are employed to obtain high-quality annotations. Through these steps, the fine-tuned model demonstrates the enhanced performance of the proposed pipeline compared to existing models. Experiments confirm the pipeline's effectiveness in preserving and integrating the unique features of ancient Chinese architecture.
The research domain is relatively narrow, focusing specifically on ancient Chinese architecture. This limited scope might pose a challenge for broader acceptance at a conference like ICLR, despite the seemingly good performance within the specific domain.
We introduce a  Image-Annotation-Augmented Diffusion pipeline , enhancing image generation of the Chinese ancient architecture. We leverage SAM2 for image processing and BLIP3, RAG and GraphRAG to integrate architectural and cultural elements.
@inproceedings{
anonymous2024preserving,
title={Preserving the Unique Heritage of Chinese Ancient Architecture in Diffusion Models with Text and Image Integration},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RsmIgTLt9e},
note={under review}
}
Leveraging the impressive generative capabilities of diffusion models, we can create diverse images from imaginative prompts with careful design. To be noticed, the key components, such as CLIP, are essential for aligning prompts with image representations. However, these models often underperform in specialized areas, like the Chinese ancient architecture. One of the important reasons is that historical buildings include not only architectural information, but also historical and cultural content. The preservation and integration of these unique characteristics has become a significant challenge in model expansion. In this paper, we propose an Image-Annotation-Augmented Diffusion pipeline combining human feedback to explore the specific-area paradigm for image generation in the context of small amounts of data and professional concepts. We first leverage Segment Anything 2 (SAM2) to obtain a refined content image to enable an in-depth analysis of the relationship between unique characteristics and multimodal image generation models, and reselected representative images and regrouped them according to their distinctive objective and the existing dataset. Then, we introduce the effective RAG and GraphRAG module to identify the complex structure of relationships among different entities in the training and inference stages respectively. Based on the initial text by BLIP3, the RAG instructs GPT4 to facilitate more accurate, content-aware annotations during training, and augment a high-quality object prompt using the GraphRAG during inference. Benefit from these outstanding models and architectures, we train fine-tuning models to showcase the enhanced performance of our proposed pipeline compared to other existing models. Experiments demonstrate that our pipeline effectively preserves and integrates the unique characteristics of ancient Chinese architecture.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Diffusion RAG Representation Learning Chinese Ancient Architecture
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/4a24a78071ca123f2a693ba8870db8ee94608f89.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/c79925b1e142bdbefc4ab560aa4ed9956ebea1fa.pdf
Preserving the Unique Heritage of Chinese Ancient Architecture in Diffusion Models with Text and Image Integration
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission