Update README.md
Browse files
README.md
CHANGED
|
@@ -14,15 +14,17 @@ Produced using [AutoFP8 with calibration samples from ultrachat](https://github.
|
|
| 14 |
|
| 15 |
## Evaluation
|
| 16 |
|
|
|
|
|
|
|
| 17 |
### Open LLM Leaderboard evaluation scores
|
| 18 |
-
| |
|
| 19 |
| :------------------: | :----------------------: | :------------------------------------------------: |
|
| 20 |
-
| arc-c<br>25-shot |
|
| 21 |
-
| hellaswag<br>10-shot |
|
| 22 |
-
| mmlu<br>5-shot |
|
| 23 |
-
| truthfulqa<br>0-shot |
|
| 24 |
-
| winogrande<br>5-shot |
|
| 25 |
-
| gsm8k<br>5-shot |
|
| 26 |
-
| **Average<br>Accuracy** | **79.
|
| 27 |
-
| **Recovery** | **100%** | **
|
| 28 |
|
|
|
|
| 14 |
|
| 15 |
## Evaluation
|
| 16 |
|
| 17 |
+
74.53666667 69.19 82.49 70.61 65.73 82.63 76.57
|
| 18 |
+
|
| 19 |
### Open LLM Leaderboard evaluation scores
|
| 20 |
+
| | Mixtral-8x22B-Instruct-v0.1 | Mixtral-8x22B-Instruct-v0.1-FP8<br>(this model) |
|
| 21 |
| :------------------: | :----------------------: | :------------------------------------------------: |
|
| 22 |
+
| arc-c<br>25-shot | 72.70 | 69.19 |
|
| 23 |
+
| hellaswag<br>10-shot | 89.08 | 82.49 |
|
| 24 |
+
| mmlu<br>5-shot | 77.77 | 70.61 |
|
| 25 |
+
| truthfulqa<br>0-shot | 68.14 | 65.73 |
|
| 26 |
+
| winogrande<br>5-shot | 85.16 | 82.63 |
|
| 27 |
+
| gsm8k<br>5-shot | 82.03 | 76.57 |
|
| 28 |
+
| **Average<br>Accuracy** | **79.15** | **74.53** |
|
| 29 |
+
| **Recovery** | **100%** | **94.17%** |
|
| 30 |
|