DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
synthetic data generation;diffusion models;language model
Yes
No ethics review needed.
Please see comments above
- DiffLM introduces a combination of VAEs, diffusion models, and LLMs for synthetic data generation. The incorporation of a latent diffusion model within the VAE framework, coupled with the soft-prompt injection mechanism, distinguishes DiffLM from prior work that primarily focuses on either fine-tuning LLMs or using simpler latent variable models for text generation.
- The paper provides a thorough evaluation across a diverse set of structured datasets and tasks (tabular, code, and tool generation), demonstrating the robustness and adaptability of the proposed framework.
- The observation that synthetic data generated by DiffLM can outperform real data on certain downstream tasks is  compelling and suggests a potential for knowledge enhancement through synthetic data.
- The proposed DiffLM framework addresses a significant challenge in leveraging LLMs for data synthesis, namely, controlling the generated data's structure and distribution. The ability to generate high-quality synthetic data for structured formats has broad implications for various applications, including data augmentation, privacy preservation, and software testing.
This paper introduces DiffLM, a novel framework for controllable synthetic data generation using large language models (LLMs). DiffLM addresses the challenges of LLM-based data synthesis, such as limited understanding of target data distributions and complex prompt engineering, particularly for structured data. The key innovation lies in decoupling the learning of data distributions from the LLM's training objectives. This is achieved by employing a variational autoencoder (VAE) to learn a latent representation of the real data, which is then used to guide the LLM's generation process. To enhance the quality of the learned latent space, a diffusion model is incorporated, mitigating the limitations of traditional VAEs in capturing complex distributions. Furthermore, a soft prompt injection module seamlessly integrates the learned latent information into the LLM decoding process without retraining, preserving the LLM's inherent knowledge and reasoning abilities. The authors evaluate DiffLM on seven real-world structured datasets spanning tabular, code, and tool data. Their experiments demonstrate that DiffLM generates high-quality synthetic data, achieving comparable or even superior performance to existing methods on downstream tasks, and even surpassing the performance of real data in certain cases. The proposed framework offers a flexible and robust approach for controllable data synthesis, paving the way for wider adoption of LLMs in various data generation scenarios.
- This work suggests that the proposed approach can achieve controllable synthetic data generation, but the model didnt use any disentangled latent variables learning or other grounding techniques to achieve such controllability, can the authors clarify what exactly the controllability refers to?
- In experiments section the works shows that the synthetic data can outperform real data for continued pre-training, which is intriguing and interesting, it would be more helpful to provide more analysis and empirical insights on why and how this is made possible.
- How does the interpolation of latent space look like? How and where did the diffusion model improve the issues of regular VAE model as described in the main text?
- While automated metrics like DCR and downstream task performance are useful, they do not fully capture the nuances of data quality, especially for complex structured data. Incorporating human evaluation, particularly for code and tool generation, would provide a more holistic assessment of the generated data's usability and realism.
Yes
No ethics review needed.
1. Is the injector module tuned in Figure 1? Clarity on this would be beneficial.
2. In Table 1, should the column "Beijing MLE" with a down arrow actually be an up arrow?
3. Which specific LLM is utilized in DiffLM presented in Table 1?
4. How many data samples were synthesized for each of the tasks evaluated?
5. What is the data volume used specifically for tool-related tasks?
6. How does the diversity of synthetic data impact downstream results? Can you control the generation diveristy of synthetic data?  An exploration of this aspect would be informative.
1. The paper addresses the significant and relevant application of generative models: generating high-quality synthetic textual and structural data, which is an important field to study.
2. DiffLM is the first framework to combine diffusion models with LLMs for the purpose of high-quality data synthesis.
The paper presents DiffLM, a novel framework for synthetic data generation that combines LLMs with diffusion models. The primary goal is to address the challenges of generating high-quality synthetic data with a specific focus on structured formats such as tabular, code, and tool data. DiffLM introduces a plug-and-play latent feature injection module to decouple the learning of target distribution knowledge from the generative objectives of LLMs. The framework is evaluated on seven real-world datasets, demonstrating that DiffLM can outperform real data in certain downstream tasks.
1. In Table 1, DiffLM does not consistently outperform TabSyn across various columns. While the authors compare state-of-the-art results based on language models, there are concerns:
- GReaT is based on GPT-2, which is a relatively weak baseline. A more robust baseline, such as GPT-4 or at least Mistral 7B, should be considered.
- Since TabSyn is also a diffusion-based table generation method, the integration of LLMs in DiffLM does not significantly enhance performance compared to TabSyn, suggesting that DiffLM may be a less effective solution for table generation.
2. The paper claims to focus on data generation with only a small set of real-world distribution data. However, for table and code data, the dataset size (~24k) is substantial. Consequently, specialized models like TabSyn have sufficient training data to perform well without LLM assistance, which diminishes the perceived advantage of DiffLM.
3. The study does not explore how the quantity of real & synthetic data affects downstream task performance respectively, which is crucial for understanding the practical impact of synthetic data.

In conclusion, certain details remain unclear in the paper (see questions), and some settings are not fully addressed.
Yes
No ethics review needed.
1. For different data, do we need to train different VAE encoder? If yes, can we still call it as plug-and-play module?
2. As we know, diffusion models are good at generating diverse samples. So how to guarantee the generated samples are consistent? ( for exmaples, sometimes there will be some mismatches in the generated samples)
3. Besides training a VAE based on LLMs, what do you think is the major contribution of this work?
4. Besides specific data format, is it possible to use this method to do natural text generation? Will the generated text make sence?
1. DiffLM effectively captures real-world data distributions through a combination of VAE and diffusion models, which enables the generation of high-quality synthetic data.
2. By using soft prompt injection, DiffLM allows for easy control over generated data characteristics without retraining the language model, which makes it adaptable across diverse data types and tasks.
3. DiffLM can generate various structured formats data.
The paper introduces a novel framework, named DiffLM, for controllable synthetic data generation using diffusion models and LLMs. It addresses challenges in synthetic data generation by combining a variational autoencoder (VAE) with a latent diffusion module to enhance the understanding of target data distributions. The model was tested across different type of datasets, including tabular, code, and tool data. Results showed that DiffLM could surpass real data performance in some cases.
1. Some very related works are missing. There are some works that also combine PLMs, VAE and diffusion. The authors should do a concrete survey.
2. The contribution is limited in my opinion. I think the main contribution: the authors align a encoder for LLM and do the experiments on the task, since doing generation in the latent space of VAE by diffusion model is quite common.
Yes
No ethics review needed.
1. Line 153 mentions that $D \cap D_{syn} = \emptyset$, yet it is unclear to me how Figure 5 demonstrates this condition is met for the proposed method but not for LLM fine-tuning.
2. For the fine-tuning baseline, the results may be highly dependent on training parameters (e.g., number of epochs, memorization, sampling strategy). What steps were taken to ensure a fair comparison?


**References**

1. Kaiser and Bengio. "Discrete Autoencoders for Sequence Models."
2. Amani et al. "Symbolic Autoencoding for Self-Supervised Sequence Learning."
3. Higgins et al. "$\beta$-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework."
4. Havrylov and Titov. "Preventing Posterior Collapse with Levenshtein Variational Autoencoder."
5. Bowman et al. "Generating sentences from a continuous space".
The paper is structured clearly, addressing an important problem in synthetic data generation.

Experimental results indicate performance improvements, and the visualizations are well-crafted, aiding comprehension of the methods and outcomes.

Ablation studies on injection methods provide useful insights
This paper investigates methods for generating synthetic data samples from a pre-trained large language model (LLM) using diffusion-based methods to approximate the underlying data distribution. The authors explore the limitations of standard fine-tuning approaches in capturing true data distribution and propose an alternative involving a VAE (Variational Autoencoder) setup where a latent representation encodes the distribution and a conditioned LLM decoder generates samples.

The approach can be summarized as follows:

1. Fine-tuning an LLM to generate samples often results in samples diverging from the original data distribution (cited in lines 90-96).
2. To address this, the authors propose using a VAE with a learned latent representation to capture the data distribution, conditioning the LLM decoder on these representations through a soft prompt generated by an MLP.
3. Standard VAE setups can face issues with strong decoders, where the encoder vector may be ignored. To mitigate this, the authors introduce denoising diffusion steps in the training and employ a $\beta$-VAE for additional control over the KL-divergence term via the adjustable $\beta$ parameter.

Experiments span three task types across seven datasets.
The work lacks a comprehensive ablation and stronger baseline comparisons. Although the authors conduct ablations, they focus on minor details like the $\beta$ adjustments and design choice of latent feature injection, rather than the main components of the proposed pipeline. Key design choices lack justification and comparative analysis, for instance the choice of doing denoising diffusion as I describe later. Despite references to the limitations of standard fine-tuning with sampling, the paper does not provide a thorough comparison of these methods against the proposed framework, showing enough evidence on what precise problem of finetuning and sampling is resolved (please check the questions for more on this.)

There is also a lack of evidence for key claims. In Lines 54-55, the paper mentions that standard VAE representations tend to be ignored by the LLM, driving the need for the diffusion-based approach. However, there is insufficient evidence or citation support for this assertion. Similar issues have been addressed in prior works (e.g., [2, 4, 5]).

Similarly in lines 195-196, if **the core issue** is a discrepancy between the encoder’s posterior and prior, citations or empirical evidence are needed to support this claim. Additionally, simpler solutions, such as adopting a more expressive prior distribution (e.g., Mixture of Gaussians), could have been discussed as alternatives to diffusion. I think the choice of diffusion models is not justified in the current write up.

**Some Minor Recommendations and Editorial Notes**

- I recommend citing the original $\beta$-VAE work since you're using this technique (reference [3]).
- **Conditional Generation with LLMs and VAEs**: Since the paper explores variational autoencoding with LLMs, you could consider to reference prior work on conditional generation with hard and soft tokens in LLMs with auto-encoders. Related work such as [1, 2, 4, 5] would provide readers with a broader context and highlight parallels to existing methods.
DiffLM combines VAEs and diffusion models with LLMs to generate high-quality synthetic data via a novel latent feature injection method.
@inproceedings{
anonymous2024difflm,
title={Diff{LM}: Controllable Synthetic Data Generation via Diffusion Language Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=fRmfDqZ2yq},
note={under review}
}
Recent advancements in large language models (LLMs) have significantly enhanced their knowledge and generative capabilities, leading to a surge of interest in leveraging LLMs for high-quality data synthesis. However, synthetic data generation via prompting LLMs remains challenging due to LLMs' limited understanding of target data distributions and the complexity of prompt engineering, especially for structured formatted data. To address these issues, we introduce DiffLM, a controllable data synthesis framework based on variational autoencoder (VAE), which further (1) leverages diffusion models to reserve more information of original distribution and format structure in the learned latent distribution and (2) decouples the learning of target distribution knowledge from the LLM's generative objectives via a plug-and-play latent feature injection module. As we observed significant discrepancies between the VAE's latent representations and the real data distribution, the latent diffusion module is introduced into our framework to learn a fully expressive latent distribution. Evaluations on seven real-world datasets with structured formatted data (i.e., Tabular, Code and Tool data) demonstrate that DiffLM generates high-quality data, with performance on downstream tasks surpassing that of real data by 2\%–7\% in certain cases. Data and code will be released upon acceptance.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
synthetic data generation diffusion models language model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7b0eb4a25c756f0a302078e8aeeba56dbf47d58b.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission