README.md

---
base_model: onekq-ai/OneSQL-v0.1-Qwen-7B
tags:
- text-generation-inference
- transformers
- qwen2
- awq
license: apache-2.0
language:
- en
---

# Introduction

This model is the AWQ version of [OneSQL-v0.1-Qwen-7B](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-7B).

# Performances

The self-evaluation EX score of the original model is **56.19** (compared to **63.33** by the 32B model on the [BIRD leaderboard](https://bird-bench.github.io/).
The self-evaluation EX score of this AWQ model is **47.84**.

# Quick start

To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**.
Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit.

```python
from vllm import LLM, SamplingParams

llm = LLM(model="onekq-ai/OneSQL-v0.1-Qwen-7B-AWQ")
sampling_params = SamplingParams(temperature=0.7, max_tokens=200)

prompt="CREATE TABLE students (
    id INTEGER PRIMARY KEY,
    name TEXT,
    age INTEGER,
    grade TEXT
);

-- Find the three youngest students
SELECT "

outputs = llm.generate(f"<|im_start|>system\nYou are a SQL expert. Return code only.<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n", sampling_params)
print(outputs[0].outputs[0].text.strip())
```

The model response is the finished SQL query without **SELECT**
```sql
* FROM students ORDER BY age ASC LIMIT 3
```

# Caveats

The performance drop from the original model is due to quantization itself, and the lack of beam search support in the vLLM framework. Use at your own discretion.