---
# 🦄 Model Card
base_model: unsloth/Qwen2.5-3B-Instruct
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
- grpo   # Gradient Reward Policy Optimization
license: apache-2.0
language:
- en
---

# 📦 Uploaded Model  

| **Field**             | **Value**                                  |
|-----------------------|--------------------------------------------|
| **Developed by**      | **bhaviktheslider**                        |
| **License**           | Apache 2.0                                 |
| **Finetuned from**    | `unsloth/Qwen2.5-3B-Instruct`              |
| **Training Framework**| [Unsloth](https://github.com/unslothai/unsloth) × Hugging Face TRL |

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

---

## 🚀 What’s New?

> **TL;DR** – Think of this model as the beefed-up, protein-shake-powered sequel to **MasterControlAIML/DeepSeek-R1-Qwen2.5-1.5b-SFT-R1-JSON-Unstructured-To-Structured** … except we ditched the SFT and let a squad of reward functions do the coaching.

### Key Upgrades
1. **Larger Backbone** – We jumped from a 1.5 B parameter model to a 3 B parameter **Qwen 2.5** variant for more representational oomph.  
2. **No SFT, All 🍬 Rewards** – Instead of supervised fine-tuning, training relied solely on reward-based optimization (GRPO).  
   - **LM-as-Judge**: A language model scored candidate outputs for task quality.  
   - **Auxiliary Rewards**: Style, length, and JSON-validity rewards kept the model on its best behavior.  
3. **2× Faster Training** – Courtesy of Unsloth’s memory-efficient tricks (flash attention + fused optimizers).  

---

## 🛠️ Intended Use

- Converts messy, free-form text into structured JSON—exactly like its 1.5 B predecessor, but with a deeper knowledge reservoir and reinforcement-tuned precision.  
- Drop-in replacement for any pipeline already using the DeepSeek-R1 model. Just swap checkpoints and enjoy the headroom.

---

## 🏋️ Training Details

| Item | Value |
|------|-------|
| **Base Model** | `unsloth/Qwen2.5-3B-Instruct` |
| **Batching** | Gradient Accumulation 8, bfloat16 |
| **Optimizer** | AdamW (fused) |
| **Algorithm** | GRPO (policy ≈ LM; reward model ≈ separate LM judge) |
| **Epochs** | 3 (effective) |
| **Speed** | ~2× faster vs. vanilla PyTorch thanks to Unsloth |

---

## 📊 Evaluation (Coming Soon)

We’re benchmarking against:
- Exact-match JSON accuracy  
- Structural F1  
- Valid-JSON rate  

…stay tuned—numbers arriving faster than you can say “schema validation.”

---

## 🤝 Citation

If you build something cool with this model, a shout-out would be lovely:

```bibtex
@misc{bhaviktheslider_2025_unsloth_qwen2.5_3b_grpo,
  title  = {An Unsloth-accelerated GRPO-trained Qwen 2.5 3B for JSON structuring},
  author = {Bhaviktheslider},
  year   = {2025},
  howpublished = {Hugging Face},
  note   = {https://huggingface.co/bhaviktheslider/<repo>}
}