gobean commited on
Commit
ed482f9
·
verified ·
1 Parent(s): 26ed0c7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -5
README.md CHANGED
@@ -1,11 +1,14 @@
1
  ---
2
  license: apache-2.0
3
  ---
4
- Coming soon! just learned about thebloke's quant issues, will update later.
5
 
 
6
 
 
 
 
7
 
8
- This is a llamafile for [Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
9
 
10
 
11
 
@@ -22,6 +25,6 @@ doesn't work, the file might be named something else so I had success with
22
  If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
23
 
24
 
25
- Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems.
26
- You just run it, open the chat interface in a browser, and interact.
27
- Options can be passed in to expose the api etc.
 
1
  ---
2
  license: apache-2.0
3
  ---
 
4
 
5
+ This is a llamafile for [Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
6
 
7
+ These were converted and quantized from source safetensors using llama.cpp on April 3, 2024.
8
+ This matters because there are several GGUF files on HF which were created before llama.cpp's support for MoE quantization was fully debugged,
9
+ even though it looked like it was producing working files at the time.
10
 
11
+ I'll be uploading the quantized .gguf sources I created as well if anyone wants them as a reference or for further work.
12
 
13
 
14
 
 
25
  If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
26
 
27
 
28
+ Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems including FreeBSD, Windows, Windows via WSL, Linux, and Mac.
29
+ The same file works everywhere. You just download, run it, open the chat interface in a browser, and interact. Options can be passed in to expose the api etc.
30
+ See their docs for details.