Update README.md

README.md

@@ -1,9 +1,42 @@
 ---
 library_name: transformers
-tags: []
+tags:
+- reward
+- RM
+- Code
+- AceCode
+- AceCoder
+license: mit
+datasets:
+- TIGER-Lab/AceCode-89K
+- TIGER-Lab/AceCodePair-300K
+language:
+- en
+base_model:
+- Qwen/Qwen2.5-Coder-7B-Instruct
 ---
+# 🂡 AceCoder
 
-## Uses
+[Paper](#) | 
+[Github](https://github.com/TIGER-AI-Lab/AceCoder) |
+[AceCode-89K](https://huggingface.co/datasets/TIGER-Lab/AceCode-89K) |
+[AceCodePair-300K](https://huggingface.co/datasets/TIGER-Lab/AceCodePair-300K) |
+[RM/RL Models](https://huggingface.co/collections/TIGER-Lab/acecoder-67a16011a6c7d65cad529eba)
+
+We introduce AceCoder, the first work to propose a fully automated pipeline for synthesizing large-scale reliable tests used for the reward model training and reinforcement learning in the coding scenario. To do this, we curated the dataset AceCode-89K, where we start from a seed code dataset and prompt powerful LLMs to "imagine" proper test cases for the coding question and filter the noisy ones. We sample inferences from existing coder models and compute their pass rate as the reliable and verifiable rewards for both training the reward model and conducting the reinforcement learning for coder LLM.
+
+**This model is the official AceCodeRM-7B trained from Qwen2.5-Coder-7B-Instruct on [TIGER-Lab/AceCodePair-300K](https://huggingface.co/datasets/TIGER-Lab/AceCodePair-300K)**
+
+![https://tiger-ai-lab.github.io/AceCoder/static/images/ac_overview.png](https://tiger-ai-lab.github.io/AceCoder/static/images/ac_overview.png)
+
+
+## Performance on Best-of-N sampling
+
+![https://tiger-ai-lab.github.io/AceCoder/static/images/ac_table2.png](https://tiger-ai-lab.github.io/AceCoder/static/images/ac_table2.png)
+
+## Usage
+
+- To use the RM to produce rewards, please apply the following example codes:
 
 ```python
 import torch
@@ -151,3 +184,17 @@ chosen_scores = values.gather(
 chosen_scores = chosen_scores.squeeze()
 print(chosen_scores)
 ```
+
+
+- To use the RM for the RL tuning, please refer to our [Github Code](https://github.com/TIGER-AI-Lab/AceCoder) for more details
+
+## Citation
+```bibtex
+@article{AceCoder,
+    title={AceCoder: Acing Coder RL via Automated Test-Case Synthesis},
+    author={Zeng, Huaye and Jiang, Dongfu and Wang, Haozhe and Nie, Ping and Chen, Xiaotong and Chen, Wenhu},
+    journal={ArXiv},
+    year={2025},
+    volume={abs/2207.01780}
+}
+```