parameters guide
samplers guide
model generation
role play settings
quant selection
arm quants
iq quants vs q quants
optimal model setting
gibberish fixes
coherence
instructing following
quality generation
chat settings
quality settings
llamacpp server
llamacpp
lmstudio
sillytavern
koboldcpp
backyard
ollama
model generation steering
steering
model generation fixes
text generation webui
ggufs
exl2
full precision
quants
imatrix
neo imatrix
llama
llama-3
gemma
gemma2
gemma3
llama-2
llama-3.1
llama-3.2
mistral
Mixture of Experts
mixture of experts
mixtral
Update README.md
Browse files
README.md
CHANGED
@@ -16,16 +16,20 @@ These settings / suggestions can be applied to all models including GGUF, EXL2,
|
|
16 |
It also includes critical settings for Class 3 and Class 4 models at this repo - DavidAU - to enhance and control generation
|
17 |
for specific as a well as outside use case(s) including role play, chat and other use case(s).
|
18 |
|
19 |
-
This settings can also fix a number of model issues such as:
|
20 |
|
21 |
- "Gibberish"
|
22 |
-
-
|
23 |
-
-
|
|
|
|
|
24 |
- instruction following
|
25 |
- creativeness or lack there of or .. too much - purple prose.
|
26 |
- low quant (ie q2k, iq1s, iq2s) issues.
|
|
|
|
|
27 |
|
28 |
-
Likewise setting can also improve model generation and/or general overall "smoothness" / "quality" of model operation.
|
29 |
|
30 |
Even if you are not using my models, you may find this document useful for any model available online.
|
31 |
|
@@ -209,6 +213,8 @@ PENALITY SAMPLERS:
|
|
209 |
|
210 |
These samplers "trim" or "prune" output in real time. The longer the generation, the stronger overall effect.
|
211 |
|
|
|
|
|
212 |
PRIMARY:
|
213 |
|
214 |
<B>repeat-last-n</B>
|
@@ -308,7 +314,7 @@ mirostat_eta: 0.1 is a good value.
|
|
308 |
|
309 |
|
310 |
This is the big one ; activating this will help with creative generation. It can also help with stability. Also note which
|
311 |
-
samplers are
|
312 |
|
313 |
This is both a sampler (and pruner) and enhancement all in one.
|
314 |
|
@@ -369,8 +375,7 @@ Suggest you experiment with this one, with other advanced samplers disabled to s
|
|
369 |
<B>l, logit-bias TOKEN_ID(+/-)BIAS </B>
|
370 |
|
371 |
modifies the likelihood of token appearing in the completion,
|
372 |
-
|
373 |
-
or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
|
374 |
|
375 |
This may or may not be available. This requires a bit more work.
|
376 |
|
@@ -383,28 +388,6 @@ I suggest you get some "bad outputs" ; get the "tokens" (actual number for the "
|
|
383 |
Careful testing is required, as this can have unclear side effects.
|
384 |
|
385 |
|
386 |
-
------------------------------------------------------------------------------
|
387 |
-
OTHER:
|
388 |
-
------------------------------------------------------------------------------
|
389 |
-
|
390 |
-
|
391 |
-
<B>-s, --seed SEED </B>
|
392 |
-
|
393 |
-
RNG seed (default: -1, use random seed for -1)
|
394 |
-
|
395 |
-
<B>samplers SAMPLERS </B>
|
396 |
-
|
397 |
-
samplers that will be used for generation in the order, separated by ';' (default: top_k;tfs_z;typ_p;top_p;min_p;xtc;temperature)
|
398 |
-
|
399 |
-
<B>sampling-seq SEQUENCE </B>
|
400 |
-
|
401 |
-
simplified sequence for samplers that will be used (default: kfypmxt)
|
402 |
-
|
403 |
-
<B>ignore-eos </B>
|
404 |
-
|
405 |
-
ignore end of stream token and continue generating (implies --logit-bias EOS-inf)
|
406 |
-
|
407 |
-
|
408 |
------------------------------------------------------------------------------
|
409 |
ADVANCED SAMPLERS:
|
410 |
------------------------------------------------------------------------------
|
|
|
16 |
It also includes critical settings for Class 3 and Class 4 models at this repo - DavidAU - to enhance and control generation
|
17 |
for specific as a well as outside use case(s) including role play, chat and other use case(s).
|
18 |
|
19 |
+
This settings can also fix a number of model issues (any model) such as:
|
20 |
|
21 |
- "Gibberish"
|
22 |
+
- Generation length (including out of control generation)
|
23 |
+
- Chat quality.
|
24 |
+
- Letter, word, phrase, paragraph repeats
|
25 |
+
- Coherence
|
26 |
- instruction following
|
27 |
- creativeness or lack there of or .. too much - purple prose.
|
28 |
- low quant (ie q2k, iq1s, iq2s) issues.
|
29 |
+
- general output quality.
|
30 |
+
- role play related issues.
|
31 |
|
32 |
+
Likewise ALL the setting below can also improve model generation and/or general overall "smoothness" / "quality" of model operation.
|
33 |
|
34 |
Even if you are not using my models, you may find this document useful for any model available online.
|
35 |
|
|
|
213 |
|
214 |
These samplers "trim" or "prune" output in real time. The longer the generation, the stronger overall effect.
|
215 |
|
216 |
+
CLASS 4: For these models it is important to activate / set all samplers as noted for maximum quality and control.
|
217 |
+
|
218 |
PRIMARY:
|
219 |
|
220 |
<B>repeat-last-n</B>
|
|
|
314 |
|
315 |
|
316 |
This is the big one ; activating this will help with creative generation. It can also help with stability. Also note which
|
317 |
+
samplers are disabled/ignored here, and that "mirostat_eta" is a learning rate.
|
318 |
|
319 |
This is both a sampler (and pruner) and enhancement all in one.
|
320 |
|
|
|
375 |
<B>l, logit-bias TOKEN_ID(+/-)BIAS </B>
|
376 |
|
377 |
modifies the likelihood of token appearing in the completion,
|
378 |
+
i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello', or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
|
|
|
379 |
|
380 |
This may or may not be available. This requires a bit more work.
|
381 |
|
|
|
388 |
Careful testing is required, as this can have unclear side effects.
|
389 |
|
390 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
391 |
------------------------------------------------------------------------------
|
392 |
ADVANCED SAMPLERS:
|
393 |
------------------------------------------------------------------------------
|