Intel
/

Mistral-7B-Instruct-v0.2-int4-inc

Model card Files Files and versions Community

yintongl commited on May 21, 2024

Commit

b3fa32f

·

verified ·

1 Parent(s): 007ef32

Update README.md

Files changed (1) hide show

README.md +11 -12

README.md CHANGED Viewed

@@ -46,18 +46,17 @@ lm_eval --model hf --model_args pretrained="Intel/Mistral-7B-Instruct-v0.2-int4-
 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.6647 | 0.6621 |
-| mmlu           | 0.5906 | 0.5872 |
-| lambada_openai | 0.7141 | 0.7141 |
-| hellaswag      | 0.6602 | 0.6557 |
-| winogrande     | 0.7395 | 0.7364 |
-| piqa           | 0.8052 | 0.8047 |
-| truthfulqa_mc1 | 0.5251 | 0.5153 |
-| openbookqa     | 0.3600 | 0.3420 |
-| boolq          | 0.8535 | 0.8541 |
-| rte            | 0.7040 | 0.7148 |
-| arc_easy       | 0.8161 | 0.8165 |
-| arc_challenge  | 0.5435 | 0.5435 |

 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.6591 | 0.6573 |
+| mmlu           | 0.5877 | 0.5866 |
+| lambada_openai | 0.7155 | 0.7089 |
+| hellaswag      | 0.6602 | 0.6567 |
+| winogrande     | 0.7411 | 0.7395 |
+| piqa           | 0.8014 | 0.8074 |
+| truthfulqa_mc1 | 0.5251 | 0.5116 |
+| openbookqa     | 0.3520 | 0.3420 |
+| boolq          | 0.8529 | 0.8526 |
+| arc_easy       | 0.8136 | 0.8144 |
+| arc_challenge  | 0.5418 | 0.5538 |