Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,8 @@ language:
|
|
14 |
# Pantheon-RP-1.5-12b-Nemo
|
15 |
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
|
16 |
|
|
|
|
|
17 |
## Model details
|
18 |
This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first;
|
19 |
|
@@ -25,21 +27,22 @@ This time around I went for a multi-stage finetuning process as Mistral Nemo was
|
|
25 |
|
26 |
## Inference
|
27 |
|
28 |
-
|
29 |
```
|
30 |
-
"temperature": 1.0,
|
31 |
"repetition_penalty": 1.05,
|
32 |
"top_p": 0.95
|
33 |
"top_k": 40
|
34 |
"min_p": 0.05
|
35 |
```
|
|
|
36 |
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
|
37 |
|
38 |
**Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
|
39 |
|
40 |
## General Roleplay
|
41 |
|
42 |
-
The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style (aka Markdown), as that is the style my Pantheon Roleplay dataset uses.
|
43 |
|
44 |
There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts.
|
45 |
|
|
|
14 |
# Pantheon-RP-1.5-12b-Nemo
|
15 |
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
|
16 |
|
17 |
+
**Disclaimer:** Despite my goal to create the perfect Pantheon finetune I still feel like I've been unable to shave some rougher edges off the Nemo base model. Rather then continue to bash my head against the wall (and as a result not release anything) I've instead decided to release my finest attempt so far as it should already surpass my 1.0 release.
|
18 |
+
|
19 |
## Model details
|
20 |
This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first;
|
21 |
|
|
|
27 |
|
28 |
## Inference
|
29 |
|
30 |
+
Nemo is a somewhat strange model when it comes to temperatures so I highly encourage you to experiment to see which works best.
|
31 |
```
|
32 |
+
"temperature": 0.3-1.0,
|
33 |
"repetition_penalty": 1.05,
|
34 |
"top_p": 0.95
|
35 |
"top_k": 40
|
36 |
"min_p": 0.05
|
37 |
```
|
38 |
+
|
39 |
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
|
40 |
|
41 |
**Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
|
42 |
|
43 |
## General Roleplay
|
44 |
|
45 |
+
The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style (aka Markdown), as that is the style my Pantheon Roleplay dataset uses. I expect there to be a bias inside the model itself towards responding in this style.
|
46 |
|
47 |
There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts.
|
48 |
|