Update README.md
Browse files
README.md
CHANGED
@@ -49,7 +49,9 @@ AWQ is an efficient, accurate and blazing-fast low-bit weight quantization metho
|
|
49 |
|
50 |
These are experimental first AWQs for the brand-new model format, Mistral.
|
51 |
|
52 |
-
|
|
|
|
|
53 |
|
54 |
<!-- description end -->
|
55 |
<!-- repositories-available start -->
|
@@ -66,7 +68,6 @@ They will not work from vLLM or TGI. They can only be used from AutoAWQ, and the
|
|
66 |
|
67 |
```
|
68 |
<s>[INST] {prompt} [/INST]
|
69 |
-
|
70 |
```
|
71 |
|
72 |
<!-- prompt-template end -->
|
|
|
49 |
|
50 |
These are experimental first AWQs for the brand-new model format, Mistral.
|
51 |
|
52 |
+
As of September 29th 2023, they are supported by AutoAWQ, and vLLM (version 0.2).
|
53 |
+
|
54 |
+
To use from AutoAW£Q requires installing both AutoAWQ and Transformers from Github. More details are below.
|
55 |
|
56 |
<!-- description end -->
|
57 |
<!-- repositories-available start -->
|
|
|
68 |
|
69 |
```
|
70 |
<s>[INST] {prompt} [/INST]
|
|
|
71 |
```
|
72 |
|
73 |
<!-- prompt-template end -->
|