Josephgflowers/TinyLlama-R1-LIMO

Model Overview

TinyLlama-R1-LIMO is a small, efficient transformer-based model designed to improve mathematical reasoning with minimal but high-quality training data. It was fine-tuned on the LIMO dataset, which emphasizes the principle that "Less Is More" for reasoning tasks. The model is part of ongoing research to enhance instruction-following and reasoning capabilities using a dataset of only 817 curated samples.

Model Name: Josephgflowers/Tinyllama-R1-LIMO-Agent

This model was made possible by the generous support of www.cherryrepublic.com

---

Key Features

• Data Efficiency: Achieves competitive reasoning performance using the LIMO dataset with only 817 training samples.
• Mathematical Reasoning Focus: Tailored for tasks requiring logical and numerical problem-solving.
• Instruction Adaptation: The model shows improved chain-of-thought (CoT) reasoning but may require further refinement for handling complex, multi-step prompts.
• Training Pipeline: Built using the LLaMA-Factory framework with dataset-specific optimizations.

---

Model Details

• Model Type: Transformer-based (TinyLlama architecture)
• Parameter Count: 1.1B
• Training Framework: Unsloth 8k context / Hugging Face Transformers
• Primary Use Cases:
  • Mathematical and logical reasoning
  • STEM education and problem-solving
  • Instruction-following conversations

---

Training Data

This model was fine-tuned using the LIMO dataset, which emphasizes the power of high-quality data over quantity.

Dataset Highlights

• Name: LIMO (Less Is More for Reasoning)
• Size: 817 samples

Acknowledgments

Thanks to the creators of the LIMO dataset and contributors to the LLaMA-Factory training framework. Special thanks to Joseph Flowers for model fine-tuning and experimentation. Citation

If you use this model or dataset, please cite the following paper:

@misc{ye2025limoreasoning, title={LIMO: Less is More for Reasoning}, author={Yixin Ye and Zhen Huang and Yang Xiao and Ethan Chern and Shijie Xia and Pengfei Liu}, year={2025}, eprint={2502.03387}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2502.03387}, }