Update README.md
Browse files
README.md
CHANGED
@@ -2,10 +2,28 @@ Quantization w/ imatrix of Replete-AI/Mistral-11b-v0.1
|
|
2 |
|
3 |
The 'groups_merged.txt' in the repo was used to generate the 'imatrix.dat'also in the repo w/ 350 importance matrix entries.
|
4 |
|
5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
|
|
|
|
|
7 |
main: build = 2354 (e25fb4b1) SYCL, GGML_SYCL_F16: yes
|
8 |
main: built with Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 (2024.0.2.20231213) for x86_64-unknown-linux-gnu
|
9 |
-
|
|
|
10 |
2x Intel(R) Xeon(R) Platinum 8480+
|
11 |
-
4x Intel MAX 1100 GPU
|
|
|
2 |
|
3 |
The 'groups_merged.txt' in the repo was used to generate the 'imatrix.dat'also in the repo w/ 350 importance matrix entries.
|
4 |
|
5 |
+
Quants:
|
6 |
+
```
|
7 |
+
3.3G Mistral-11b-v0.1_IQ2_XXS.gguf
|
8 |
+
4.3G Mistral-11b-v0.1_Q2_K.gguf
|
9 |
+
5.5G Mistral-11b-v0.1_Q3_K.gguf
|
10 |
+
5.9G Mistral-11b-v0.1_Q3_K_L.gguf
|
11 |
+
5.0G Mistral-11b-v0.1_Q3_K_S.gguf
|
12 |
+
6.3G Mistral-11b-v0.1_Q4_0.gguf
|
13 |
+
7.0G Mistral-11b-v0.1_Q4_1.gguf
|
14 |
+
6.7G Mistral-11b-v0.1_Q4_K.gguf
|
15 |
+
6.4G Mistral-11b-v0.1_Q4_K_S.gguf
|
16 |
+
7.6G Mistral-11b-v0.1_Q5_0.gguf
|
17 |
+
8.2G Mistral-11b-v0.1_Q5_1.gguf
|
18 |
+
7.6G Mistral-11b-v0.1_Q5_K_S.gguf
|
19 |
+
9.0G Mistral-11b-v0.1_Q6_K.gguf
|
20 |
+
```
|
21 |
|
22 |
+
llama.cpp build info:
|
23 |
+
```
|
24 |
main: build = 2354 (e25fb4b1) SYCL, GGML_SYCL_F16: yes
|
25 |
main: built with Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 (2024.0.2.20231213) for x86_64-unknown-linux-gnu
|
26 |
+
```
|
27 |
+
```
|
28 |
2x Intel(R) Xeon(R) Platinum 8480+
|
29 |
+
4x Intel MAX 1100 GPU```
|