Hugging Face's logo

OsamaMo
/
Arabic_Text-To-SQL_using_Qwen2.5-1.5B

Text2Text Generation
Transformers
Safetensors
Arabic
Text-To-SQL
Arabic
Spider
SQL
Inference Endpoints
Model card Files Community
1

Model Card for Arabic Text-To-SQL (OsamaMo)

Model Details

Model Description

This model is fine-tuned on the Spider dataset with Arabic-translated questions for the Text-To-SQL task. It is based on Qwen/Qwen2.5-1.5B-Instruct and trained using LoRA on Kaggle for 15 hours on a P100 8GB GPU.

  • Developed by: Osama Mohamed (OsamaMo)
  • Funded by: Self-funded
  • Shared by: Osama Mohamed
  • Model type: Text-to-SQL fine-tuned model
  • Language(s): Arabic (ar)
  • License: MIT
  • Finetuned from: Qwen/Qwen2.5-1.5B-Instruct

Model Sources

Uses

Direct Use

This model is intended for converting Arabic natural language questions into SQL queries. It can be used for database querying in Arabic-speaking applications.

Downstream Use

Can be fine-tuned further for specific databases or Arabic dialect adaptations.

Out-of-Scope Use

  • The model is not intended for direct execution of SQL queries.
  • Not recommended for non-database-related NLP tasks.

Bias, Risks, and Limitations

  • The model might generate incorrect or non-optimized SQL queries.
  • Bias may exist due to dataset translations and model pretraining data.

Recommendations

  • Validate generated SQL queries before execution.
  • Ensure compatibility with specific database schemas.

How to Get Started with the Model

Load Model 

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import re

device = "cuda" if torch.cuda.is_available() else "cpu"
base_model_id = "Qwen/Qwen2.5-1.5B-Instruct"
finetuned_model_id = "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B"

# Load the base model and adapter for fine-tuning
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)
model.load_adapter(finetuned_model_id)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)

def generate_resp(messages):
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([text], return_tensors="pt").to(device)
    generated_ids = model.generate(
        model_inputs.input_ids,
        max_new_tokens=1024,
        do_sample=False,  temperature= False,
    )
    generated_ids = [
        output_ids[len(input_ids):]
        for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]
    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response

Example Usage 


# Production-ready system message for SQL generation
system_message = (
    "You are a highly advanced Arabic text-to-SQL converter. Your mission is to Understand first the db schema and reltions between it and then accurately transform Arabic "
    "natural language queries into SQL queries with precision and clarity.\n"
)

def get_sql_query(db_schema, arabic_query):
    # Construct the instruction message including the DB schema and the Arabic query
    instruction_message = "\n".join([
        "## DB-Schema:",
        db_schema,
        "",
        "## User-Prompt:",
        arabic_query,
        "# Output SQL:",
        "```SQL"
    ])
    
    messages = [
        {"role": "system", "content": system_message},
        {"role": "user", "content": instruction_message}
    ]
    
    response = generate_resp(messages)
    
    # Extract the SQL query from the response using a regex to capture text within the ```sql markdown block
    match = re.search(r"```sql\s*(.*?)\s*```", response, re.DOTALL | re.IGNORECASE)
    if match:
        sql_query = match.group(1).strip()
        return sql_query
    else:
        return response.strip()

# Example usage:
example_db_schema = r'''{
    'Pharmcy': 
          CREATE TABLE `purchase` (
            `BARCODE` varchar(20) NOT NULL,
            `NAME` varchar(50) NOT NULL,
            `TYPE` varchar(20) NOT NULL,
            `COMPANY_NAME` varchar(20) NOT NULL,
            `QUANTITY` int NOT NULL,
            `PRICE` double NOT NULL,
            `AMOUNT` double NOT NULL,
            PRIMARY KEY (`BARCODE`),
            KEY `fkr3` (`COMPANY_NAME`),
            CONSTRAINT `fkr3` FOREIGN KEY (`COMPANY_NAME`) REFERENCES `company` (`NAME`) ON DELETE CASCADE ON UPDATE CASCADE
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `sales` (
            `BARCODE` varchar(20) NOT NULL,
            `NAME` varchar(50) NOT NULL,
            `TYPE` varchar(10) NOT NULL,
            `DOSE` varchar(10) NOT NULL,
            `QUANTITY` int NOT NULL,
            `PRICE` double NOT NULL,
            `AMOUNT` double NOT NULL,
            `DATE` varchar(15) NOT NULL
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `users` (
            `ID` int NOT NULL,
            `NAME` varchar(50) NOT NULL,
            `DOB` varchar(20) NOT NULL,
            `ADDRESS` varchar(100) NOT NULL,
            `PHONE` varchar(20) NOT NULL,
            `SALARY` double NOT NULL,
            `PASSWORD` varchar(20) NOT NULL,
            PRIMARY KEY (`ID`)
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `history_sales` (
            `USER_NAME` varchar(20) NOT NULL,
            `BARCODE` varchar(20) NOT NULL,
            `NAME` varchar(50) NOT NULL,
            `TYPE` varchar(10) NOT NULL,
            `DOSE` varchar(10) NOT NULL,
            `QUANTITY` int NOT NULL,
            `PRICE` double NOT NULL,
            `AMOUNT` double NOT NULL,
            `DATE` varchar(15) NOT NULL,
            `TIME` varchar(20) NOT NULL
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `expiry` (
            `PRODUCT_NAME` varchar(50) NOT NULL,
            `PRODUCT_CODE` varchar(20) NOT NULL,
            `DATE_OF_EXPIRY` varchar(10) NOT NULL,
            `QUANTITY_REMAIN` int NOT NULL
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `drugs` (
            `NAME` varchar(50) NOT NULL,
            `TYPE` varchar(20) NOT NULL,
            `BARCODE` varchar(20) NOT NULL,
            `DOSE` varchar(10) NOT NULL,
            `CODE` varchar(10) NOT NULL,
            `COST_PRICE` double NOT NULL,
            `SELLING_PRICE` double NOT NULL,
            `EXPIRY` varchar(20) NOT NULL,
            `COMPANY_NAME` varchar(50) NOT NULL,
            `PRODUCTION_DATE` date NOT NULL,
            `EXPIRATION_DATE` date NOT NULL,
            `PLACE` varchar(20) NOT NULL,
            `QUANTITY` int NOT NULL,
            PRIMARY KEY (`BARCODE`)
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          CREATE TABLE `company` (
            `NAME` varchar(50) NOT NULL,
            `ADDRESS` varchar(50) NOT NULL,
            `PHONE` varchar(20) NOT NULL,
            PRIMARY KEY (`NAME`)
          ) ENGINE=InnoDB DEFAULT CHARSET=latin1

          Answer the following questions about this schema:
}'''

example_arabic_query = "اريد الباركود الخاص بدواء يبداء اسمه بحرف 's'"

sql_result = get_sql_query(example_db_schema, example_arabic_query)
print("استعلام SQL الناتج:")
print(sql_result)

Training Details

Training Data

  • Dataset: Spider (translated into Arabic)
  • Preprocessing: Questions converted to Arabic while keeping SQL queries unchanged.
  • Training format:
    • System instruction guiding Arabic-to-SQL conversion.
    • Database schema provided for context.
    • Arabic user queries mapped to correct SQL output.
    • Output is strictly formatted SQL queries enclosed in markdown code blocks.

Training Procedure

Training Hyperparameters

  • Batch size: 1 (per device)
  • Gradient accumulation: 4 steps
  • Learning rate: 1.0e-4
  • Epochs: 3
  • Scheduler: Cosine
  • Warmup ratio: 0.1
  • Precision: bf16

Speeds, Sizes, Times

  • Training time: 15 hours on NVIDIA P100 8GB
  • Checkpointing every: 500 steps

Evaluation

Testing Data

  • Validation dataset: Spider validation set (translated to Arabic)

Metrics

  • Exact Match (EM) for SQL correctness
  • Execution Accuracy (EX) on databases

Results

  • Model achieved competitive SQL generation accuracy for Arabic queries.
  • Further testing required for robustness.

Environmental Impact

  • Hardware Type: NVIDIA Tesla P100 8GB
  • Hours used: 15
  • Cloud Provider: Kaggle
  • Carbon Emitted: Estimated using ML Impact Calculator

Technical Specifications

Model Architecture and Objective

  • Transformer-based Qwen2.5-1.5B architecture.
  • Fine-tuned for Text-to-SQL task using LoRA.

Compute Infrastructure

  • Hardware: Kaggle P100 GPU (8GB VRAM)
  • Software: Python, Transformers, LLaMA-Factory, Hugging Face Hub

Citation

If you use this model, please cite:

@misc{OsamaMo_ArabicSQL,
  author = {Osama Mohamed},
  title = {Arabic Text-To-SQL Model},
  year = {2024},
  howpublished = {\url{https://huggingface.co/OsamaMo/Arabic_Text-To-SQL}}
}

Model Card Contact

For questions, contact Osama Mohamed via Hugging Face (OsamaMo).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Text2Text Generation
This model is not currently available via any of the supported Inference Providers.

Model tree for OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B

Base model

Qwen/Qwen2.5-1.5B
Finetuned
Qwen/Qwen2.5-1.5B-Instruct
Finetuned
(399)
this model