Update README.md

README.md

@@ -154,6 +154,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
     </tr>
     <tr>
       <th>Hardware</th>
+      <th>Number of GPUs</th>
       <th>Model</th>
       <th>Average Cost Reduction</th>
       <th>Latency (s)</th>
@@ -166,7 +167,8 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
   </thead>
   <tbody>
     <tr>
-      <td>A100x4</td>
+      <th rowspan="3" valign="top">A100</th>
+      <td>4</td>
       <td>Qwen/Qwen2-VL-72B-Instruct</td>
       <td></td>
       <td>6.5</td>
@@ -177,7 +179,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>113</td>
     </tr>
     <tr>
-      <td>A100x2</td>
+      <td>2</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w8a8</td>
       <td>1.85</td>
       <td>7.2</td>
@@ -188,7 +190,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>211</td>
     </tr>
     <tr>
-      <td>A100x1</td>
+      <td>1</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
       <td>3.32</td>
       <td>10.0</td>
@@ -199,7 +201,8 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>419</td>
     </tr>
     <tr>
-      <td>H100x4</td>
+      <th rowspan="3" valign="top">H100</td>
+      <td>4</td>
       <td>Qwen/Qwen2-VL-72B-Instruct</td>
       <td></td>
       <td>4.4</td>
@@ -210,7 +213,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>99</td>
     </tr>
     <tr>
-      <td>H100x2</td>
+      <td>2</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-FP8-Dynamic</td>
       <td>1.79</td>
       <td>4.7</td>
@@ -221,7 +224,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>177</td>
     </tr>
     <tr>
-      <td>H100x1</td>
+      <td>1</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
       <td>2.60</td>
       <td>6.4</td>
@@ -233,7 +236,10 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
     </tr>
   </tbody>
 </table>
- 
+
+**Use case profiles: Image Size (WxH) / prompt tokens / generation tokens
+
+**QPD: Queries per dollar, based on on-demand cost at [Lambda Labs](https://lambdalabs.com/service/gpu-cloud) (observed on 2/18/2025).
 
 ### Multi-stream asynchronous performance (measured with vLLM version 0.7.2)
 
@@ -261,7 +267,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
   </thead>
   <tbody>
     <tr>
-      <td>A100x4</td>
+      <th rowspan="3" valign="top">A100x4</th>
       <td>Qwen/Qwen2-VL-72B-Instruct</td>
       <td></td>
       <td>0.3</td>
@@ -272,29 +278,27 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>595</td>
     </tr>
     <tr>
-      <td>A100x2</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w8a8</td>
       <td>1.84</td>
-      <td>0.6</td>
+      <td>1.2</td>
       <td>293</td>
-      <td>2.0</td>
+      <td>4.0</td>
       <td>1021</td>
-      <td>2.3</td>
+      <td>4.6</td>
       <td>1135</td>
     </tr>
     <tr>
-      <td>A100x1</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
       <td>2.73</td>
-      <td>0.6</td>
+      <td>2.4</td>
       <td>314</td>
-      <td>3.2</td>
+      <td>12.8</td>
       <td>1591</td>
-      <td>4.0</td>
+      <td>16.0</td>
       <td>2019</td>
     </tr>
     <tr>
-      <td>H100x4</td>
+      <th rowspan="3" valign="top">H100x4</td>
       <td>Qwen/Qwen2-VL-72B-Instruct</td>
       <td></td>
       <td>0.5</td>
@@ -305,27 +309,31 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>377</td>
     </tr>
     <tr>
-      <td>H100x2</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-FP8-Dynamic</td>
       <td>1.70</td>
-      <td>0.8</td>
+      <td>1.6</td>
       <td>236</td>
-      <td>2.2</td>
+      <td>4.4</td>
       <td>623</td>
-      <td>2.4</td>
+      <td>4.8</td>
       <td>669</td>
     </tr>
     <tr>
-      <td>H100x1</td>
       <td>neuralmagic/Qwen2-VL-72B-Instruct-quantized.w4a16</td>
       <td>2.35</td>
-      <td>1.3</td>
+      <td>5.2</td>
       <td>350</td>
-      <td>3.3</td>
+      <td>13.2</td>
       <td>910</td>
-      <td>3.6</td>
+      <td>14.4</td>
       <td>994</td>
     </tr>
   </tbody>
 </table>
+
+**Use case profiles: Image Size (WxH) / prompt tokens / generation tokens
+
+**QPS: Queries per second.
+
+**QPD: Queries per dollar, based on on-demand cost at [Lambda Labs](https://lambdalabs.com/service/gpu-cloud) (observed on 2/18/2025).