Intel
/

Qwen1.5-0.5B-Chat-int4-inc

Model card Files Files and versions Community

yintongl commited on May 21, 2024

Commit

59f5abf

·

verified ·

1 Parent(s): 54542f8

Update README.md

Files changed (1) hide show

README.md +10 -10

README.md CHANGED Viewed

@@ -53,17 +53,17 @@ lm_eval --model hf --model_args pretrained="Intel/Qwen1.5-0.5B-Chat-int4-inc",au
 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.3879 | 0.3848 |
-| mmlu           | 0.3013 | 0.2911 |
-| lambada_openai | 0.4106 | 0.4002 |
-| hellaswag      | 0.3621 | 0.3601 |
-| winogrande     | 0.5517 | 0.5430 |
-| piqa           | 0.6757 | 0.6632 |
-| truthfulqa_mc1 | 0.2546 | 0.2485 |
-| openbookqa     | 0.1900 | 0.1880 |
 | boolq          | 0.3994 | 0.4379 |
-| arc_easy       | 0.4823 | 0.4697 |
-| arc_challenge  | 0.2517 | 0.2466 |

 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.3896 | 0.3848 |
+| mmlu           | 0.3162 | 0.2911 |
+| lambada_openai | 0.4137 | 0.4002 |
+| hellaswag      | 0.3630 | 0.3601 |
+| winogrande     | 0.5509 | 0.5430 |
+| piqa           | 0.6730 | 0.6632 |
+| truthfulqa_mc1 | 0.2583 | 0.2485 |
+| openbookqa     | 0.1880 | 0.1880 |
 | boolq          | 0.3994 | 0.4379 |
+| arc_easy       | 0.4848 | 0.4697 |
+| arc_challenge  | 0.2491 | 0.2466 |