Update README.md
Browse files
README.md
CHANGED
@@ -11,39 +11,14 @@ base_model: THUDM/chatglm3-6b
|
|
11 |
|
12 |
# chatglm3-6b-awq-w-int4-asym-gs128-a-fp16-onnx-ryzen-strix-hybrid
|
13 |
- ## Introduction
|
14 |
-
|
15 |
-
|
16 |
-
- Postprocess
|
17 |
- ## Quantization Strategy
|
18 |
- AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
|
19 |
- Excluded Layers: None
|
20 |
-
```
|
21 |
-
python3 quantize_quark.py \
|
22 |
-
--model_dir "$model" \
|
23 |
-
--output_dir "$output_dir" \
|
24 |
-
--quant_scheme w_uint4_per_group_asym \
|
25 |
-
--num_calib_data 128 \
|
26 |
-
--quant_algo awq \
|
27 |
-
--dataset pileval_for_awq_benchmark \
|
28 |
-
--seq_len 512 \
|
29 |
-
--model_export quark_safetensors \
|
30 |
-
--data_type float16 \
|
31 |
-
--exclude_layers [] \
|
32 |
-
--custom_mode awq
|
33 |
-
```
|
34 |
-
- ## OGA Model Builder
|
35 |
-
```
|
36 |
-
python builder.py \
|
37 |
-
-i <quantized safetensor model dir> \
|
38 |
-
-o <oga model output dir> \
|
39 |
-
-p int4 \
|
40 |
-
-e dml
|
41 |
-
```
|
42 |
-
- PostProcessed to generate Hybrid Model
|
43 |
-
-
|
44 |
-
- ## Quick Start
|
45 |
-
For quickstart, refer to hybrid-llm-artifacts_1.3.0.zip available in [RyzenAI-SW-EA](https://account.amd.com/en/member/ryzenai-sw-ea.html)
|
46 |
|
|
|
|
|
47 |
|
48 |
#### Evaluation scores
|
49 |
The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 29.7801.
|
|
|
11 |
|
12 |
# chatglm3-6b-awq-w-int4-asym-gs128-a-fp16-onnx-ryzen-strix-hybrid
|
13 |
- ## Introduction
|
14 |
+
This model was prepared using the AMD Quark Quantization tool, followed by necessary post-processing.
|
15 |
+
|
|
|
16 |
- ## Quantization Strategy
|
17 |
- AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
|
18 |
- Excluded Layers: None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
|
20 |
+
- ## Quick Start
|
21 |
+
For quickstart, refer to [Ryzen AI doucmentation](https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html)
|
22 |
|
23 |
#### Evaluation scores
|
24 |
The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 29.7801.
|