Update README.md
Browse files
README.md
CHANGED
@@ -1,11 +1,14 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
-
Coming soon! just learned about thebloke's quant issues, will update later.
|
5 |
|
|
|
6 |
|
|
|
|
|
|
|
7 |
|
8 |
-
|
9 |
|
10 |
|
11 |
|
@@ -22,6 +25,6 @@ doesn't work, the file might be named something else so I had success with
|
|
22 |
If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
|
23 |
|
24 |
|
25 |
-
Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems.
|
26 |
-
You just run it, open the chat interface in a browser, and interact.
|
27 |
-
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
|
|
4 |
|
5 |
+
This is a llamafile for [Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
|
6 |
|
7 |
+
These were converted and quantized from source safetensors using llama.cpp on April 3, 2024.
|
8 |
+
This matters because there are several GGUF files on HF which were created before llama.cpp's support for MoE quantization was fully debugged,
|
9 |
+
even though it looked like it was producing working files at the time.
|
10 |
|
11 |
+
I'll be uploading the quantized .gguf sources I created as well if anyone wants them as a reference or for further work.
|
12 |
|
13 |
|
14 |
|
|
|
25 |
If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
|
26 |
|
27 |
|
28 |
+
Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems including FreeBSD, Windows, Windows via WSL, Linux, and Mac.
|
29 |
+
The same file works everywhere. You just download, run it, open the chat interface in a browser, and interact. Options can be passed in to expose the api etc.
|
30 |
+
See their docs for details.
|