Update README.md
Browse files
README.md
CHANGED
@@ -18,16 +18,17 @@ base_model:
|
|
18 |
|
19 |
[ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
|
20 |
|
21 |
-
**CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for
|
22 |
|
23 |
## Evaluations
|
24 |
|
25 |
-
ProX models are evaluated
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
-
| | ArC-c | ARC-e | CSQA | HellaS | MMLU | OBQA | PiQA | SIQA | WinoG | SciQ | AVG |
|
28 |
-
|-----------------------|-------|-------|-------|-----------|-------|-------|-------|-------|-------|-------|------|
|
29 |
-
| raw | 26.1 | 44.3 | 29.7 | 39.1 | 27.3 | 29.2 | 66.9 | 39.0 | 52.0 | 67.4 | 42.1 |
|
30 |
-
| ours | 26.4 | 51.9 | 30.9 | 42.4 | 29.4 | 31.6 | 67.9 | 40.0 | 52.2 | 73.5 | 44.6 |
|
31 |
|
32 |
### Citation
|
33 |
```
|
|
|
18 |
|
19 |
[ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
|
20 |
|
21 |
+
**CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for **10**B tokens.
|
22 |
|
23 |
## Evaluations
|
24 |
|
25 |
+
ProX models are evaluated on 9 common math reasoning benchmarks.
|
26 |
+
|
27 |
+
| Model | asdiv | gsm8k | mathqa | mawps | minerva_math | mmlu_stem | sat_math | svamp | tabmwp | average |
|
28 |
+
|----------------------- |---------- |---------- |---------- |---------- |-------------- |----------- |---------- |---------- |---------- |---------- |
|
29 |
+
| CodeLlama-7B | 50.7 | 11.8 | 14.3 | 62.6 | 5.0 | 20.4 | 21.9 | 44.2 | 30.6 | 29.1 |
|
30 |
+
| CodeLlama-7B-ProXMath | **67.9** | **35.6** | **38.9** | **82.7** | **17.6** | **42.6** | **62.5** | **55.8** | **41.3** | **49.4** |
|
31 |
|
|
|
|
|
|
|
|
|
32 |
|
33 |
### Citation
|
34 |
```
|