Update README.md
Browse files
README.md
CHANGED
@@ -2,4 +2,21 @@
|
|
2 |
license: cc-by-nc-4.0
|
3 |
base_model:
|
4 |
- Qwen/Qwen2.5-Coder-32B
|
5 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
license: cc-by-nc-4.0
|
3 |
base_model:
|
4 |
- Qwen/Qwen2.5-Coder-32B
|
5 |
+
---
|
6 |
+
|
7 |
+
# ExCoT
|
8 |
+
|
9 |
+
Snowflake’s AI research team introduces ExCoT, the first model in the Arctic Text2SQL family. ExCoT is a novel framework that combines CoT prompting with SQL execution-based DPO, using execution results — not human preferences — as the feedback signal. This enables scalable, high-quality model optimization without requiring expensive human annotations.
|
10 |
+
|
11 |
+
Based on our internal testing, ExCoT delivered state-of-the-art results on the [BIRD-test benchmark](https://bird-bench.github.io/), achieving best-in-class performance in the single-model, single-inference category using only public datasets (BIRD and Spider) and no additional Text2SQL data:
|
12 |
+
|
13 |
+
* Arctic-ExCoT-70B (based on LLaMA 3.1 70B) improved execution accuracy on the BIRD-dev set from the base model’s 57.37% to 68.51%. Arctic-ExCoT-32B (based on Qwen-2.5-Coder 32B) achieved similarly strong gains.
|
14 |
+
|
15 |
+
* Both models significantly outperformed larger open-weight models, such as Mistral 123B, and even proprietary systems including GPT-4o and Claude 3.5 — achieving over 12 percentage points of improvement.
|
16 |
+
|
17 |
+
For more details about ExCoT and how to use it:
|
18 |
+
|
19 |
+
* ❄️ [Arctic Text2SQL: Introducing ExCoT for Execution-Guided Chain-of-Thought Optimization (blog)]()
|
20 |
+
* 📝 [ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback (arxiv)](https://arxiv.org/pdf/2503.19988)
|
21 |
+
* 🚀 [Getting started guide using ArcticTraining](https://github.com/snowflakedb/ArcticTraining/tree/main/projects/excot_dpo)
|
22 |
+
|