Text Generation
GGUF
English
mixture of experts
Mixture of Experts
4x8B
Llama3 MOE
creative
creative writing
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prosing
vivid writing
fiction
roleplaying
bfloat16
swearing
rp
horror
mergekit
Inference Endpoints
conversational
Update README.md
Browse files
README.md
CHANGED
|
@@ -124,6 +124,9 @@ You can set the number of experts in LMStudio (https://lmstudio.ai) at the "load
|
|
| 124 |
|
| 125 |
For Text-Generation-Webui (https://github.com/oobabooga/text-generation-webui) you set the number of experts at the loading screen page.
|
| 126 |
|
|
|
|
|
|
|
|
|
|
| 127 |
For server.exe / Llama-server.exe (Llamacpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md )
|
| 128 |
add the following to the command line to start the "llamacpp server" (CLI):
|
| 129 |
|
|
|
|
| 124 |
|
| 125 |
For Text-Generation-Webui (https://github.com/oobabooga/text-generation-webui) you set the number of experts at the loading screen page.
|
| 126 |
|
| 127 |
+
For KolboldCPP (https://github.com/LostRuins/koboldcpp) Version 1.8+ , on the load screen, click on "TOKENS",
|
| 128 |
+
you can set experts on this page, and the launch the model.
|
| 129 |
+
|
| 130 |
For server.exe / Llama-server.exe (Llamacpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md )
|
| 131 |
add the following to the command line to start the "llamacpp server" (CLI):
|
| 132 |
|