satreysa commited on
Commit
d03d57f
·
verified ·
1 Parent(s): 76a97ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -29
README.md CHANGED
@@ -11,39 +11,14 @@ base_model: THUDM/chatglm3-6b
11
 
12
  # chatglm3-6b-awq-w-int4-asym-gs128-a-fp16-onnx-ryzen-strix-hybrid
13
  - ## Introduction
14
- - Quantization Tool: Quark 0.6.0
15
- - OGA Model Builder: v0.5.1
16
- - Postprocess
17
  - ## Quantization Strategy
18
  - AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
19
  - Excluded Layers: None
20
- ```
21
- python3 quantize_quark.py \
22
- --model_dir "$model" \
23
- --output_dir "$output_dir" \
24
- --quant_scheme w_uint4_per_group_asym \
25
- --num_calib_data 128 \
26
- --quant_algo awq \
27
- --dataset pileval_for_awq_benchmark \
28
- --seq_len 512 \
29
- --model_export quark_safetensors \
30
- --data_type float16 \
31
- --exclude_layers [] \
32
- --custom_mode awq
33
- ```
34
- - ## OGA Model Builder
35
- ```
36
- python builder.py \
37
- -i <quantized safetensor model dir> \
38
- -o <oga model output dir> \
39
- -p int4 \
40
- -e dml
41
- ```
42
- - PostProcessed to generate Hybrid Model
43
- -
44
- - ## Quick Start
45
- For quickstart, refer to hybrid-llm-artifacts_1.3.0.zip available in [RyzenAI-SW-EA](https://account.amd.com/en/member/ryzenai-sw-ea.html)
46
 
 
 
47
 
48
  #### Evaluation scores
49
  The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 29.7801.
 
11
 
12
  # chatglm3-6b-awq-w-int4-asym-gs128-a-fp16-onnx-ryzen-strix-hybrid
13
  - ## Introduction
14
+ This model was prepared using the AMD Quark Quantization tool, followed by necessary post-processing.
15
+
 
16
  - ## Quantization Strategy
17
  - AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
18
  - Excluded Layers: None
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
+ - ## Quick Start
21
+ For quickstart, refer to [Ryzen AI doucmentation](https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html)
22
 
23
  #### Evaluation scores
24
  The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 29.7801.