Update README.md
Browse files
README.md
CHANGED
@@ -16,8 +16,8 @@ In benchmarking, we observe **no accuracy loss** and up to **30\%** performance
|
|
16 |
## 1. Benchmarking Result (detailed in [PULL REQUEST](https://github.com/sgl-project/sglang/pull/3730)):
|
17 |
| Model | Config | Accuracy (GSM8K) | Accuracy (MMLU) | Output Throughput(qps=128) | Output Throughput(bs=1) |
|
18 |
|--------|--------|-------------------|----------------|------------------------------|--------------------------|
|
19 |
-
|
|
20 |
-
|
|
21 |
|
22 |
## 2. Quantization Process
|
23 |
|
|
|
16 |
## 1. Benchmarking Result (detailed in [PULL REQUEST](https://github.com/sgl-project/sglang/pull/3730)):
|
17 |
| Model | Config | Accuracy (GSM8K) | Accuracy (MMLU) | Output Throughput(qps=128) | Output Throughput(bs=1) |
|
18 |
|--------|--------|-------------------|----------------|------------------------------|--------------------------|
|
19 |
+
| INT8 R1 | (A100\*16)x2 | 95.8 | 87.1 | 4450.02 (+33%) | 44.18 (+18%) |
|
20 |
+
| BF16 R1 | A100\*32 | 95.5 | 87.1 | 3342.29 | 37.20 |
|
21 |
|
22 |
## 2. Quantization Process
|
23 |
|