Decoupling Layout from Glyph in Online Chinese Handwriting Generation
Online handwriting generation; Layout generation; Calligraphy imitation; Conditional diffusion Model
> Comment 3:  Section 4.3.2 lacks quantitative experiments.

**Answer**: Actually, our calligraphy classifier **can also achieves a classification accuracy of 91%** for the entire generated text line. The reason for this is not explicitly included in the paper is that, in Chinese, calligraphy style can be predominantly reflected at the level of individual characters. Since our method generates each character independently, the quantitative evaluation of calligraphy style of the entire line **largely overlaps with** the character-level experiments presented in Section 4.2.

> Comment 4:  The generated layouts show significant absences in Figure 7. 

**Answer**: Recalling that our layout generation method requires a few characters bounding boxes as a prefix. In Figure 7, we use the first ten characters as the prefix, so the bounding boxes for these characters are the same as the real ones rather than being missing.

> Comment 5:  It is recommended to compare the proposed method with style transfer-based approaches in Table 3.

**Answer**:  Thanks for your insightful suggestion. We conducted a supplementary experiment for Table 3. by combining the layout generation module with the SDT method：
|   | AR   | CR   |
| ------- | ------- | ------- |
| In-context SDT | 0.871  | 0.868 |

As mentioned before, our method consists of two parts: layout generation and character generation. The hierarchical framework makes our approach highly flexible. In the character generation component, we can use either a data-driven or style transfer approach, which is completely **decoupled from the layout generation part**. Thus, we believe that the comparison between these methods is also largely overlaps with the experiments in Section 4.2.  Therefore, in the text line generation experiments, we primarily focus more on evaluating the **effectiveness of the layout generator**, via both quantitative evaluation through binary geometric features and qualitative evaluation through visualization.

> Comment 6: About the applicability and what will happen if some simple layout extraction methods are used to extract the pseudo-layouts of style references.

**Answer**:  1) For customized handwritten text generation, the user needs to provide a few reference characters. In practical applications, we only require the user to write a small piece of text line (e.g 10 characters). The layout generator will mimic the layout characteristics (e.g., font size, letter spacing) of these reference characters when generating the layout for subsequent characters. Additionally, these 10 reference characters will serve as style references for the character generation model.
2）Performance of simple layout style extraction methods tends to be **overly dependent on** carefully designed features, such as the binary geometric features I used in Table 2. In my own experiments, I found that model-free approaches that are too simplistic often generate layouts with noticeable differences compared to real samples, such as failing to properly handle the relative positions of punctuation marks and text. Furthermore, these methods exhibit poor generalization and are **only limited to text line generation**, whereas model-based approaches can be transferred to other even 2-Dimension handwritten data, such as handwritten math equations.

> Comment 7:  Few generated visual results and lacks visual comparisons with the baseline.

**Answer**: Thank you for your suggestion. We conduct visulization and subjective experiments in Figure 1,8,9  and we will provide more additional visual results in the revised version to make our paper more convincing. However, in the domain of text line generation, this task remains largely unexplored, and as such, there is a lack of established baselines. Our approach represents a novel attempt at addressing this task.

---
We sincerely appreciate your detailed and thoughtful feedback on our manuscript. We have carefully addressed each of your comments and have made the necessary revisions and additions in the official version of the manuscript.  We hope that our responces meet your expectations and would be grateful if you could consider revising the rating in light of the improvements made. We welcome any further questions or discussions you may have, and we will be happy to provide more elaborate responses as needed！
Thank you very much for your comments and questions, which are of great significance for us to improve our article and work. Next, I will respond in detail.

### **Regarding the novelty and contribution** :
Firstly, we would like to emphasize that the primary novelty of our work lies in the proposal of a hierarchical approach to **solving the challenging task of handwritten Chinese text line generation**, a problem that has been **rarely explored**. This decoupling strategy can also be effectively extended to other complex handwriting generation tasks. Our contributions within this framework are twofold: 1) A novel method for explicit layout modeling, and 2) A purely data-driven character generation approach based on a 1D U-Net. 


For 1)： I would like to highlight that we cleverly adapt the in-context generation paradigm from **next-token prediction** in LLM to the task of generating layouts. This simple yet effective approach forms the core of our method. To the best of our knowledge, our work is one of the earliest explorations into handwritten Chinese text line generation, and it is the **first to explicitly generate the layout for text line**. Additionally, an extra benefit of our approach is that the generated data naturally contains strong positional label of characters, which makes it highly convenient for data augmentation in tasks such as character segmentation and recognition.

For 2）: We design a 1D convolutional network capable of extracting multi-scale features specifically for **online sequential data**. In contrast, previous approaches for sequential handwriting data typically use LSTM or Transformer models, which do not explicitly consider multi-scale features. Furthermore, our approach is purely data-driven, distinguishing it from the style transfer paradigm commonly used in the state-of-the-art methods. These two approaches are not mutually exclusive and can complement each other in different application scenarios.

Below, we address the reviewer’s questions one by one:

---

### **Response to Specific Comments** :
> Comment 1: About the novelty of the multi-scale style encoder and multi-scale contrastive loss.

**Answer** : Thank you for pointing out the relevant literature. We will ensure that additional references are incorporated and elaborate more on the differences and complementarities between previous methods including [a],[b] and our own in the official version. It is important to note that the work referenced in [a] deals with offline handwritten data, specifically images. In contrast, all the data used in our work is online data, which is sequential in nature. Networks designed for processing **online handwritten data** typically **do not explicitly** model multi-scale features in the previous related work. In this paper, we introduce a model based on 1D-convolutional networks, marking the first instance of explicitly modeling multi-scale features in online Chinese handwritten data generation field. Additionally, the contrastive loss function we design is tightly integrated with this multi-scale network to enhance its ability to distinguish calligraphy styles from different writers at different scales. While in paper[b], the proposed writer-wise and character-wise contrastive learning is applied **only at a single feature scale**. From this perspective, our contributions are not identical, but they could even **complement each other**. 

> Comment 2:  It is not clear : 1) How the prefix guide the generation in layout generator.  2) The modality of style reference data.  3) The task setting.

**Answer** :  1) A more intuitive explanation can be drawn from the **next-token-prediction paradigm** in language models. When a partial sequence of text is input as a context, the subsequent generated content tends to be coherent with the prefix. Similarly, in our task, when the layout information for the first few characters is input as a context, the layout of subsequent characters is naturally consistent with the prefix. For example, if the initial layout exhibits a general skew, there is a higher probability that this characteristic will be maintained in the generated text, as we demonstrate in the experiment section (Figure 7).  This is what we call **“in-context-like layout generation”** .  2) As described in Section 3.1, all the data we used in this paper is online sequential data. Therefore, the novelty of our approach lies, to some extent, in its focus on designing a network specifically tailored for online handwritten data compared with previous work. 3) For fairness in comparison, we have kept the experimental setup consistent with previous methods, specifically using a few-shot setting, where 10 reference characters are used for style reference.


[a] Wang H, Wang Y, Wei H. Affganwriting: a handwriting image generation method based on multi-feature fusion, ICDAR, 2023.

[b] Dai G, Zhang Y, et al. Disentangling writer and character styles for handwriting generation, CVPR, 2023.
Response to Concerns Regarding the Novelty and the Effectiveness of the Proposed Method.
Yes
No ethics review needed.
Please see Weaknesses
1. The study proposes a hierarchical method to address the under-explored task of online handwritten Chinese text line generation.
2. By decoupling layout generation from glyph generation, the method offers more flexibility in handling the generation of text lines, which is particularly useful when dealing with complex Chinese characters.
3.  The experiments conducted on the CASIA-OLHWDB database indicate high performance in imitation sample generation, demonstrating the effectiveness of the method.
The paper introduces a novel approach for generating online handwritten Chinese text with specific styles. The authors naturally divide a text line into two components: layout and glyphs, and design a text line layout generator coupled with a diffusion-based stylized font synthesizer to address this challenge hierarchically. The layout generator autoregressively generates the positions for each glyph based on text content and provided style references, while the font synthesizer generates each font at its position while imitating the calligraphy style extracted from the given style references. Experiments on the CASIAOLHWDB demonstrate the method's capability to generate structurally correct and indistinguishable imitation samples.
1. While decoupling layout and glyph generation increases flexibility, it may also add to the model's complexity, potentially affecting training and inference efficiency.
2. Are there any application scenarios for this task? The author could analyze its practicality.
3. The paper mentions difficulties in imitating styles with extensive cursive connections between characters due to the independent generation of each character, indicating potential limitations in handling certain calligraphic styles.
Yes
No ethics review needed.
(1) Could more details be provided on how the layout-glyph separation specifically enhances performance in comparison to prior models?

(2) Would additional experiments on style consistency across diverse text lines clarify the benefits of this approach?

(3) Could this method can be adapted to non-Chinese scripts or connected handwriting styles?
(1) The hierarchical decomposition into layout and glyph generation is an innovative approach, particularly suited for complex scripts like Chinese. This framework successfully addresses challenges specific to the language, such as the diversity of character structures.

(2) The model is thoroughly tested on both character and line generation, with metrics tailored to layout and stylistic fidelity. The model's success across multiple metrics shows a well-rounded, effective design.

(3) Despite the technical depth, the paper provides a good level of explanation for each module, with helpful visualizations that demonstrate layout and glyph generation separately.

(4) The method has potential applications in handwriting synthesis, digital personalization, and document augmentation, contributing a valuable approach for future research in multilingual handwriting generation.
This paper focuses on the generation of online Chinese handwriting text lines. It proposes a hierarchical approach that decouples layout generation from glyph generation. The text line layout generator arranges character positions based on text content and writing style references, while the font synthesizer generates characters with specific styles. The contributions include a novel layout generator, a 1D U-Net network for font generation, and a multi-scale style encoder. Experiments demonstrate the effectiveness of the method in generating structurally correct and stylistically similar samples.
(1) Missing qualitative comparisons with prior methods, limiting insights into this model’s advantages in style fidelity and layout accuracy.

(2) The contributions over previous approaches could be articulated more clearly, especially regarding the effectiveness of the layout-glyph separation.

(3) The organization could be refined for readability, as the methods section contains complex explanations that could benefit from clearer structuring.
Yes
No ethics review needed.
My main concerns are the novelty of the proposed multi-scale style encoder and style contrastive learning loss, and the effectiveness of the proposed layout generator. For details, please refer to the weaknesses.
1) This paper proposes a hierarchical online Chinese handwritten text line generation method. The proposed method utilizes a layout generator and a font synthesizer to produce the layouts and characters independently, then arranges the characters within the layouts to create complete text lines.

2) The proposed method achieves the best performance in purely data-driven font generation task.
This paper focuses on the generation of online Chinese handwritten text lines. The core of this method lies in decomposing text line generation into layout generation and character generation, and fill characters into the generated layouts to form complete text lines. Experiments evaluate the proposed method.
1) The multi-scale style encoder is not a new design in handwriting generation area, as a similar idea has been proposed in [a]. Besides, the proposed style contrastive learning loss is somewhat similar to the style learning loss in [b].

2) The method description is not clear: (1) In lines 233-237, it is mentioned that style reference samples are used as context prefixes, but how they guide the subsequent layout generation is unclear. (2) The paper does not specify the modality of the style references used, online data, or offline images. (3) The paper does not specify the number of style reference samples used, one-shot or few-shot.

3) Section 4.3.2 lacks quantitative experiments in terms of calligraphy styles, raising doubts about whether the proposed Multi-Scale Style Encoder can accurately extract calligraphy styles from entire text lines.

4) In the 'Conditional' row of Figure 7, the generated layouts (red boxes) show significant absences at the beginning of the text line, which raises concerns about the effectiveness of the layout generator.

5) It is recommended to compare the proposed method with style transfer-based approaches, as it can be relatively straightforward to extend this method to a style transfer setting by replacing character embedding with a CNN-based content encoder.

6) The layout generator requires real layouts of style references, which is not directly available in the application, does this limitation affect its applicability? If some simple layout extraction methods are used to extract the pseudo-layouts of style references, what impact would this have on generation performance?

7) The paper provides very few generated visual results and lacks visual comparisons with the baseline.

[a] Wang H, Wang Y, Wei H. Affganwriting: a handwriting image generation method based on multi-feature fusion, ICDAR, 2023.

[b] Dai G, Zhang Y, Wang Q, et al. Disentangling writer and character styles for handwriting generation, CVPR, 2023.
Yes
No ethics review needed.
1. If the bounding box generated by the layout model and the bounding box generated by the character model have different shapes, how should this be handled?

2. Since the method can be described as a unified probability distribution according to Equation 1, why not jointly train the two models end-to-end instead of training them separately?

3. The paper does not discuss whether the method can be applied to handwriting generation of other languages.

4. in Line 285, what does L represent? Although the authors write this is the length of the feature sequence, it is not clear what does this sequence represent? 

5. In Line 230, is the ground truth the same as the reference input in Figure 3？
1.While some work on English handwritten text line generation exists, as far as I know, no such work has been published for online Chinese text lines. Compared to English characters, Chinese characters have more complex structures and a larger number of categories, making English generation methods unsuitable for direct application to Chinese. This work proposes a method to address this task, representing a noticable contribution.

2.The method decouples text line generation into two steps—layout generation and character generation—under a unified probabilistic framework, providing a good theoretical foundation and considerable novelty.

3.The experimental section includes comprehensive comparative and visualization experiments for both layout generation and character generation, yielding convincing results.

4.The paper is well-organized and clearly written.
The paper addresses the task of generating online handwritten Chinese text lines condition on the content and style. It identifies that text lines can be divided into two components: layout and characters. The authors propose a hierarchical approach that includes a text line layout generator and a stylized font synthesizer. The layout generator uses in-context-like learning to determine the positions of each character, while the font synthesizer generates characters that imitate the calligraphic style of the provided references. The method is evaluated using the CASIA-OLHWDB dataset, demonstrating its effectiveness in producing structurally correct and indistinguishable imitation samples.
1. The assumption that character generation is independent given their positions seems too strong. Does text line style only manifest in the relative positions and sizes of individual characters? I hope the authors can give discussion on the reasonableness of this assumption and explain whether it might limit the method's ability in style learning. 

2. It is better to add sub-figure index for figure 8 and 9. It seems each of figure 8 and 9 has three sub-figures, but now their boundaries are not clear. In Figure 7, it is also suggested to identify which one is the proposed method in the paper. Of course, this is not a big issue.
A novel method for Online Chinese Text Line genration.
@inproceedings{
anonymous2024decoupling,
title={Decoupling Layout from Glyph in Online Chinese Handwriting Generation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=DhHIw9Nbl1},
note={under review}
}
Text plays a crucial role in the transmission of human civilization, and teaching machines to generate online handwritten text in various styles presents an interesting and significant challenge. However, most prior work has concentrated on generating individual Chinese fonts, leaving complete text line generation largely unexplored. In this paper, we identify that text lines can naturally be divided into two components: layout and glyphs. Based on this division, we designed a text line layout generator coupled with a diffusion-based stylized font synthesizer to address this challenge hierarchically. More concretely, the layout generator performs in-context-like learning based on the text content and the provided style references to generate positions for each glyph autoregressively. Meanwhile, the font synthesizer which consists of a character embedding dictionary, a multi-scale calligraphy style encoder and a 1D U-Net based diffusion denoiser will generate each font on its position while imitating the calligraphy style extracted from the given style references. Qualitative and quantitative experiments on the CASIA-OLHWDB demonstrate that our method is capable of generating structurally correct and indistinguishable imitation samples.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Online handwriting generation; Layout generation; Calligraphy imitation; Conditional diffusion Model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/cffe34c1f859efba2713662251e984ff3c2c20ae.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Decoupling Layout from Glyph in Online Chinese Handwriting Generation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission