---
language:
- en
tags:
- text-to-sql
- gpt2
- gpt2-medium
- nlp-to-sql
- text2sql
- sql
---
# Model Card for Model ID

<!-- The base model used for training is gpt2-medium. We finetuned it on the following dataset: b-mc2/sql-create-context -->

This is my first fine tuned LLM project.


## Prompt

query = List the creation year, name and budget of each department

f"Translate the following English question to SQL: {query}

## Output

SELECT zip_code FROM Visibility WHERE AVG(visibility) < 10

[More Information Needed]


#### Training Hyperparameters

num_train_epochs=1
per_device_train_batch_size=3
gradient_accumulation_steps=9
learning_rate=5e-5
weight_decay=0.01


## Evaluation

Step	Training Loss

500	    0.337800
1000	0.262900
1500	0.253200
2000	0.246400

{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}