shubhrapandit commited on
Commit
9d691ab
·
verified ·
1 Parent(s): efeae75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -452,21 +452,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
452
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w8a8</td>
453
  <td>1.80</td>
454
  <td>1.2</td>
455
- <td>289</td>
456
  <td>4.0</td>
457
- <td>1020</td>
458
  <td>4.6</td>
459
- <td>1133</td>
460
  </tr>
461
  <tr>
462
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
463
  <td>2.75</td>
464
  <td>2.8</td>
465
- <td>341</td>
466
  <td>12.8</td>
467
- <td>1588</td>
468
  <td>16.4</td>
469
- <td>2037</td>
470
  </tr>
471
  <tr>
472
  <th rowspan="3" valign="top">H100x4</th>
@@ -483,21 +483,21 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
483
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-FP8-Dynamic</td>
484
  <td>1.73</td>
485
  <td>1.8</td>
486
- <td>247</td>
487
  <td>4.4</td>
488
- <td>621</td>
489
  <td>4.8</td>
490
- <td>669</td>
491
  </tr>
492
  <tr>
493
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
494
  <td>8.27</td>
495
  <td>13.2</td>
496
- <td>913</td>
497
  <td>13.2</td>
498
- <td>913</td>
499
  <td>99.2</td>
500
- <td>6777</td>
501
  </tr>
502
  </tbody>
503
  </table>
 
452
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w8a8</td>
453
  <td>1.80</td>
454
  <td>1.2</td>
455
+ <td>578</td>
456
  <td>4.0</td>
457
+ <td>2040</td>
458
  <td>4.6</td>
459
+ <td>2266</td>
460
  </tr>
461
  <tr>
462
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
463
  <td>2.75</td>
464
  <td>2.8</td>
465
+ <td>1364</td>
466
  <td>12.8</td>
467
+ <td>6352</td>
468
  <td>16.4</td>
469
+ <td>8148</td>
470
  </tr>
471
  <tr>
472
  <th rowspan="3" valign="top">H100x4</th>
 
483
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-FP8-Dynamic</td>
484
  <td>1.73</td>
485
  <td>1.8</td>
486
+ <td>479</td>
487
  <td>4.4</td>
488
+ <td>1203</td>
489
  <td>4.8</td>
490
+ <td>1296</td>
491
  </tr>
492
  <tr>
493
  <td>neuralmagic/Qwen2.5-VL-72B-Instruct-quantized.w4a16</td>
494
  <td>8.27</td>
495
  <td>13.2</td>
496
+ <td>3652</td>
497
  <td>13.2</td>
498
+ <td>3652</td>
499
  <td>99.2</td>
500
+ <td>27108</td>
501
  </tr>
502
  </tbody>
503
  </table>