Update README.md

README.md

@@ -1,4 +1,81 @@
----
-license: mit
-library_name: transformers
----
+---
+license: mit
+library_name: transformers
+datasets:
+- AI-MO/NuminaMath-CoT
+- KbsdJames/Omni-MATH
+- RUC-AIBOX/STILL-3-Preview-RL-Data
+- hendrycks/competition_math
+language:
+- en
+base_model:
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+---
+
+<div align="center">
+<span style="font-family: default; font-size: 1.5em;">DeepScaleR-1.5B-Preview</span>
+<div>
+🚀 Democratizing Reinforcement Learning for LLMs 🌟
+</div>
+</div>
+
+<div align="center" style="line-height: 1;">
+  <a href="https://github.com/agentica-project/deepscaler" target="_blank" style="margin: 2px;">
+    <img alt="Code" src="https://img.shields.io/badge/DeepScaleR-000000?style=for-the-badge&logo=github&logoColor=000&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://img.shields.io/badge/Notion-%23000000.svg?style=for-the-badge&logo=notion&logoColor=white" target="_blank" style="margin: 2px;">
+    <img alt="Blog" src="https://www.notion.so/DeepScaleR-Scaling-R1-Models-with-Reinforcement-Learning-1891e65ddc7f80ad8cc6dbe0069a66fa?pvs=4" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://huggingface.co/agentica-org" target="_blank" style="margin: 2px;">
+    <img alt="Hugging Face" src="https://img.shields.io/badge/Agentica-fcd022?style=for-the-badge&logo=huggingface&logoColor=000&labelColor" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+
+</div>
+
+</div>
+
+## Model Details
+
+### Model Description
+
+
+## Training Details
+
+### Training Data
+
+
+### Training Procedure
+
+## Evaluation
+
+We report Pass@1 accuracy averaged over 16 samples for each problem.
+| Model | AIME 2024 | MATH 500 | AMC 2023 | Minerva Math | OlympiadBench | Avg. |
+|-------|-----------|-----------|-----------|--------------|---------------|------|
+| 2.5-7B-Instruct | 13.3 | 79.8 | 50.6 | 34.6 | 40.7 | 43.8 |
+| rStar-Math-7B | 26.7 | 78.4 | 47.5 | - | 47.1 | - |
+| Eurus-2-7B-PRIME | 26.7 | 79.2 | 57.8 | 38.6 | 42.1 | 48.9 |
+| Qwen2.5-7B-SimpleRL | 26.7 | 82.4 | 62.5 | <strong>39.7</strong> | 43.3 | 50.9 |
+| DeepSeek-R1-Distill-Qwen-1.5B | 28.8 | 82.8 | 62.9 | 26.5 | 43.3 | 48.9 |
+| Still-1.5B | 32.5 | 84.4 | 66.7 | 29.0 | 45.4 | 51.6 |
+| <strong>DeepScaleR-1.5B-Preview</strong> | <strong>43.1</strong> | <strong>87.8</strong> | <strong>73.6</strong> | 30.2 | - | - |
+| O1-Preview | 40.0 | 81.4 | - | - | - | - |
+
+## Acknowledgement
+
+- Our training experiments are powered by our heavily modified fork of [Verl](https://github.com/agentica-project/verl), an open-source RLHF library.
+- Our model is trained on top of [`DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
+- Our work is done as part of  [Berkeley Sky Computing Lab](https://skycomputing.berkeley.edu/) and [Berkeley AI Research](https://bair.berkeley.edu/).
+
+
+## Citation 
+
+```bibtex
+@misc{deepscaler2025,
+  title={DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL},
+  author={Michael Luo and Sijun Tan and Justin Wong and Xiaoxiang Shi and William Tang and Manan Roongta and Colin Cai and Jeffrey Luo and Tianjun Zhang and Erran Li and Raluca Ada Popa and Ion Stoica},
+  year={2025},
+  howpublished={\url{}},
+  note={Notion Blog}
+  year={2025}
+}