itsdotscience commited on
Commit
b81bbce
·
verified ·
1 Parent(s): 10ba8eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -3
README.md CHANGED
@@ -2,10 +2,28 @@ Quantization w/ imatrix of Replete-AI/Mistral-11b-v0.1
2
 
3
  The 'groups_merged.txt' in the repo was used to generate the 'imatrix.dat'also in the repo w/ 350 importance matrix entries.
4
 
5
- llama.cpp build info:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
 
 
7
  main: build = 2354 (e25fb4b1) SYCL, GGML_SYCL_F16: yes
8
  main: built with Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 (2024.0.2.20231213) for x86_64-unknown-linux-gnu
9
-
 
10
  2x Intel(R) Xeon(R) Platinum 8480+
11
- 4x Intel MAX 1100 GPU
 
2
 
3
  The 'groups_merged.txt' in the repo was used to generate the 'imatrix.dat'also in the repo w/ 350 importance matrix entries.
4
 
5
+ Quants:
6
+ ```
7
+ 3.3G Mistral-11b-v0.1_IQ2_XXS.gguf
8
+ 4.3G Mistral-11b-v0.1_Q2_K.gguf
9
+ 5.5G Mistral-11b-v0.1_Q3_K.gguf
10
+ 5.9G Mistral-11b-v0.1_Q3_K_L.gguf
11
+ 5.0G Mistral-11b-v0.1_Q3_K_S.gguf
12
+ 6.3G Mistral-11b-v0.1_Q4_0.gguf
13
+ 7.0G Mistral-11b-v0.1_Q4_1.gguf
14
+ 6.7G Mistral-11b-v0.1_Q4_K.gguf
15
+ 6.4G Mistral-11b-v0.1_Q4_K_S.gguf
16
+ 7.6G Mistral-11b-v0.1_Q5_0.gguf
17
+ 8.2G Mistral-11b-v0.1_Q5_1.gguf
18
+ 7.6G Mistral-11b-v0.1_Q5_K_S.gguf
19
+ 9.0G Mistral-11b-v0.1_Q6_K.gguf
20
+ ```
21
 
22
+ llama.cpp build info:
23
+ ```
24
  main: build = 2354 (e25fb4b1) SYCL, GGML_SYCL_F16: yes
25
  main: built with Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 (2024.0.2.20231213) for x86_64-unknown-linux-gnu
26
+ ```
27
+ ```
28
  2x Intel(R) Xeon(R) Platinum 8480+
29
+ 4x Intel MAX 1100 GPU```