Intel
/

Qwen2.5-1.5B-Instruct-int4-inc

Model card Files Files and versions Community

n1ck-guo commited on Oct 8, 2024

Commit

bd792b5

·

verified ·

1 Parent(s): f6bb1f7

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ datasets:
 ## Model Details
-This model is an int4 model with group_size 128 with quantized lm-head of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
 ## How To Use
@@ -88,7 +88,7 @@ python -m auto_round --model "Intel/Qwen2.5-1.5B-Instruct-int4-inc" --eval --eva
 ```
 | Metric         | BF16   | INT4(group_size 128)   | INT4(group_size 32)   |
-|:--------------:| :----: | :----: |:------:|
 | Avg            | 0.5646 | 0.5668 | 0.5699 |
 | mmlu           | 0.6010 | 0.5876 | 0.5924 |
 | cmmlu          | 0.6497 | 0.6146 | 0.6259 |

 ## Model Details
+This model is an int4 model with group_size 128 of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
 ## How To Use
 ```
 | Metric         | BF16   | INT4(group_size 128)   | INT4(group_size 32)   |
+|:--------------| :----: | :----: |:------:|
 | Avg            | 0.5646 | 0.5668 | 0.5699 |
 | mmlu           | 0.6010 | 0.5876 | 0.5924 |
 | cmmlu          | 0.6497 | 0.6146 | 0.6259 |