SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging
Text-to-SQL;LLM;Synthetic Data Generation;Databases
Yes
No ethics review needed.
No more questions. Please see above.
1. The method is evaluated on three SQL dialects and proves its effectiveness.
2. The author has built a text-to-sql dataset that includes SQL dialects. If this dataset is open source, it will be of certain value to research.
This paper introduces the SQL-GEN framework, which bridges the dialect gap for text-to-SQL systems by generating high-quality synthetic data for any dialect and merging dialect-specific models into a unified model using a novel Mixture of Experts (MoE) initialization method. The framework significantly improves execution accuracy on unseen real-world multi-dialect benchmarks and reduces the gap compared to large-scale human-annotated data.
1. The paper presents contributions that, while valuable, appear to have limited novelty compared to existing work in the field[1].
2. In addition, from the results, the improvement of synthetic data is not as good as that of bird train set.

[1] Yu T, Wu C S, Lin X V, et al. Grappa: Grammar-augmented pre-training for table semantic parsing[J]. arXiv preprint arXiv:2009.13845, 2020.
Yes
No ethics review needed.
- On a side note, it would be interesting to consider cross-lingual transfer methods for text-to-SQL as seen in multilingual NLP tasks, which might provide a useful baseline for comparison.
- **Originality**
    - The task of addressing dialects in text-to-SQL systems is nascently explored, and this paper makes a valuable contribution by introducing a novel MoE approach to manage multi-dialect scenarios effectively.
- **Quality and Clarity**: 
     - The paper is generally easy to follow.
     - The experiments comprehensively test SQL-GEN across multiple benchmarks and datasets and validate the performance.
In the well-studied text-to-SQL task, the authors examine handling multiple SQL dialects and propose SQL-GEN, a three-step method to adapt text-to-SQL models trained on one SQL dialect to other dialects by generating synthetic text-to-SQL samples. They also introduce a Mixture of Experts approach to merge dialect-specific SQL models into a single model for efficient cross-dialect knowledge sharing and reducing maintenance costs by selectively activating dialect-specific experts based on the input. The method is tested on BIRD and Paglia benchmarks and is evaluated against baseline datasets to validate SQL-GEN's efficacy.
- Figures, especially Figures 3 and 4, are somewhat dense and challenging to interpret. A clearer visual representation or more detailed explanations would enhance accessibility.
- The text in Figure 5 could be enlarged to improve readability, as it currently appears difficult to see.
Yes
No ethics review needed.
1. How does SQL-GEN handle dialect-specific syntax that drastically changes query logic, such as proprietary SQL functions that are unique to one dialect? It would be better to provide specific examples of how SQL-GEN handles a few key proprietary functions from different dialects, and discuss any limitations in this area.
2. What measures are in place to ensure the synthetic queries generated by SQL-GEN align closely with real-world dialect usage beyond benchmarks?
1. SQL-GEN’s synthetic data generation effectively addresses the lack of annotated data for various SQL dialects, enhancing Text-to-SQL systems' adaptability across dialects without needing human annotations. And it is the first work to discuss its important problem.
2. The moe architecture efficiently merges dialect-specific models into a unified framework, promoting knowledge transfer and improving performance across dialects, making the system resource-efficient for real-world use.
3. SQL-GEN enhances execution accuracy by up to 20% over previous methods, making Text-to-SQL models more reliable for multi-dialect databases, with potential performance improvements in specific dialects like PostgreSQL and BigQuery.
4. By reducing the dependency on human annotations and enabling effective multi-dialect handling, SQL-GEN offers a cost-efficient approach to multi-dialect Text-to-SQL training. It would be better to open-source the generative model to help the community.
The paper addresses the challenges of adapting Text-to-SQL systems across SQL dialects like BigQuery, PostgreSQL, and SQLite. Traditional Text-to-SQL models, optimized for SQLite, struggle with dialect-specific syntax variations, making translation and model adaptability challenging. SQL-GEN proposes a synthetic data generation method, SQL-GEN, which leverages dialect-specific SQL tutorials to create diverse, high-quality training samples for any SQL dialect. It introduces a Mixture of Experts (MoE) approach for merging dialect-specific models, enhancing multi-dialect capability.
1. While SQL-GEN aims to create a unified model that bridges dialect gaps, merging dialect-specific models can dilute the depth of expertise for each dialect, leading to a possible trade-off between dialect-specific expertise and generalizability. It would be better to discuss the drawback of SQL-GEN in each dialect-specific example and provide a detailed analysis of performance on dialect-specific features or edge cases for each dialect. This could help clarify if and where any expertise dilution occurs.

2. The Mixture of Experts (MoE) approach enables the merging of dialect-specific models, but it also requires substantial computational resources for training, fine-tuning, and deploying models, especially as more dialects are added. It would be better to incorporate an easy SFT baseline to compare and provide a computational resource comparison between their MoE approach and a standard fine-tuning approach for multiple dialects. This would give readers a clearer picture of the trade-offs involved.
SQL-GEN is a new framework that generates synthetic training data to improve text-to-SQL systems for different SQL dialects.  Additionally, SQL-GEN's MoE initialization method helps to unify multi-dialect capabilities in a single system.
@inproceedings{
anonymous2024sqlgen,
title={{SQL}-{GEN}: Bridging the Dialect Gap for Text-to-{SQL} Via Synthetic Data And Model Merging},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RaSLSUCKz0},
note={under review}
}
Text-to-SQL systems, that convert natural language queries into SQL programs, have seen significant progress with recent breakthroughs.  However,  these have been primarily for the SQLite dialect and adapting Text-to-SQL systems to other SQL dialects like BigQuery and PostgreSQL remains a challenge due to the diversity  in  SQL  syntaxes  and  functions,  along  with  the  high  cost  of  collecting and curating SQL-specific training data.   To this end,  we introduce SQL-GEN,a framework for generating high-quality synthetic data for any dialect guided by dialect-specific tutorials.  We demonstrate the effectiveness of SQL-GEN in creating training data to significantly improve the downstream Text-to-SQL performance for other dialects – it improves the execution accuracy by up to 20% over previous  methods,  and  reduces  the  gap  with  large-scale  human-annotated  data on unseen real world multi-dialect benchmarks.  Moreover, combining our synthetic data with human-annotated data provides additional performance boosts up to 5.6%. Towards unifying the multi-dialect capability in a single system, we also introduce a novel Mixture of Experts (MoE) initialization method that integrates dialect-specific models by merging self-attention layers and initializing the gates with dialect-specific keywords, yielding one unified and versatile model adept for multiple SQL dialects,  further enhancing performance across different SQL dialects. By leveraging shared core features of multiple dialect-specific models, our MOE demonstrated superior performance compared with models trained on individual dialects alone
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Text-to-SQL LLM Synthetic Data Generation Databases
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/2b21daef0fe1c836df8b3ebf2acba5945878116d.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission