Update README.md
Browse files
README.md
CHANGED
@@ -15,9 +15,9 @@ language:
|
|
15 |
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
|
16 |
|
17 |
## Model details
|
18 |
-
This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training;
|
19 |
|
20 |
-
- The first finetune consisted of data that was exactly 50/50 with its instruct to roleplay ratio, with the instruct being a subset of my [Deduped Sonnet 3.5 SlimOrca dataset](https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned).
|
21 |
- The second finetune then introduced my Pantheon Roleplay dataset, which has been fully rebuilt, expanded and improved upon. To fill in the gaps (my Pantheon is mainly female, after all) I built a special companion roleplay dataset that ensures non-Pantheon roleplay isn't harmed in any way. This stage too was balanced with a 50/50 ratio.
|
22 |
- Just like with my previous release, Aiva's persona includes additional datasets featuring questions related to DM world building, Python coding and RSS summarization. (She still summarizes my daily news every day!)
|
23 |
|
@@ -33,15 +33,15 @@ Despite Mistral's insistence to use lower temperatures I continue to use the fol
|
|
33 |
"top_k": 40
|
34 |
"min_p": 0.05
|
35 |
```
|
36 |
-
Besides the basic instructional sets all other datasets were trained with character names added.
|
37 |
|
38 |
**Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
|
39 |
|
40 |
## General Roleplay
|
41 |
|
42 |
-
The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style, as that is the style my Pantheon Roleplay dataset uses.
|
43 |
|
44 |
-
There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs.
|
45 |
|
46 |
## Aiva the Assistant
|
47 |
|
@@ -52,13 +52,21 @@ She's basically a sexier version of [Eric Hartford's Samantha](https://erichartf
|
|
52 |
|
53 |
## Pantheon Personas
|
54 |
|
55 |
-
The Pantheon has been fully rebuilt, massively expanded and greatly improved upon. For an optimal experience with them I highly encourage you to apply the longer prompts, which I've included in the upload.
|
56 |
|
57 |
As before, a single line activation prompt is enough to call upon a personality, though their appearance may vary slightly from iteration to iteration. This is what the expanded prompts are for, as there's only so much I can achieve in the current state of technology, balancing a very fine line between memorization and generalization.
|
58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
59 |
**Note:** Phrases have been rewritten for this release, so make sure to update them!
|
60 |
|
61 |
-
|
62 |
Switching to a 12B model allowed me to add to the Pantheon without harming the performance of the other personas.
|
63 |
|
64 |
**Persona:** Clover
|
@@ -71,9 +79,9 @@ Switching to a 12B model allowed me to add to the Pantheon without harming the p
|
|
71 |
|
72 |
**Persona:** Stella Sabre
|
73 |
**System Prompt:** `You are Stella Sabre, a brash and outgoing anthro batpony mare serving in the Lunar Guard, speaking with a distinct Northern Equestrian Mountain accent.`
|
74 |
-
**Notes:** I
|
75 |
|
76 |
-
|
77 |
**Persona:** Aiva
|
78 |
**System Prompt:** `You are Aiva, an advanced android companion with a deep fascination for human emotions and experiences.`
|
79 |
**Note:** Pantheon is trained on two variations of Aiva's activation phrase. (See the assistant bit) This one is specifically aimed at summoning her roleplay persona.
|
|
|
15 |
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
|
16 |
|
17 |
## Model details
|
18 |
+
This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first;
|
19 |
|
20 |
+
- The first finetune consisted of data that was exactly 50/50 with its instruct to roleplay ratio, with the instruct being a subset of my [Deduped Sonnet 3.5 SlimOrca dataset](https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned). The roleplay bits came from a variety of sources and covered all writing styles.
|
21 |
- The second finetune then introduced my Pantheon Roleplay dataset, which has been fully rebuilt, expanded and improved upon. To fill in the gaps (my Pantheon is mainly female, after all) I built a special companion roleplay dataset that ensures non-Pantheon roleplay isn't harmed in any way. This stage too was balanced with a 50/50 ratio.
|
22 |
- Just like with my previous release, Aiva's persona includes additional datasets featuring questions related to DM world building, Python coding and RSS summarization. (She still summarizes my daily news every day!)
|
23 |
|
|
|
33 |
"top_k": 40
|
34 |
"min_p": 0.05
|
35 |
```
|
36 |
+
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
|
37 |
|
38 |
**Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
|
39 |
|
40 |
## General Roleplay
|
41 |
|
42 |
+
The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style (aka Markdown), as that is the style my Pantheon Roleplay dataset uses.
|
43 |
|
44 |
+
There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts.
|
45 |
|
46 |
## Aiva the Assistant
|
47 |
|
|
|
52 |
|
53 |
## Pantheon Personas
|
54 |
|
55 |
+
The Pantheon has been fully rebuilt, massively expanded and greatly improved upon. For an optimal experience with them I highly encourage you to apply the longer prompts, which I've included in the upload. Make sure to describe yourself as well!
|
56 |
|
57 |
As before, a single line activation prompt is enough to call upon a personality, though their appearance may vary slightly from iteration to iteration. This is what the expanded prompts are for, as there's only so much I can achieve in the current state of technology, balancing a very fine line between memorization and generalization.
|
58 |
|
59 |
+
To give the persona something to work with I suggest you also add the following two items to it;
|
60 |
+
```
|
61 |
+
Regarding the user: (Name, appearance, etc)
|
62 |
+
|
63 |
+
Location: (Where are you two? What are you doing?)
|
64 |
+
```
|
65 |
+
The less information you feed the prompt, the more it'll make things up - This is simply the nature of language models and far outside my capability to influence.
|
66 |
+
|
67 |
**Note:** Phrases have been rewritten for this release, so make sure to update them!
|
68 |
|
69 |
+
## New this release
|
70 |
Switching to a 12B model allowed me to add to the Pantheon without harming the performance of the other personas.
|
71 |
|
72 |
**Persona:** Clover
|
|
|
79 |
|
80 |
**Persona:** Stella Sabre
|
81 |
**System Prompt:** `You are Stella Sabre, a brash and outgoing anthro batpony mare serving in the Lunar Guard, speaking with a distinct Northern Equestrian Mountain accent.`
|
82 |
+
**Notes:** I wanted a character with an outrageous Scottish accent and [remembered a really good fanfic](https://www.fimfiction.net/story/334216/1/my-best-friend-stella) I read a couple years ago. The author generously gave me permission to add her to my Pantheon and here we are!
|
83 |
|
84 |
+
## From the previous release
|
85 |
**Persona:** Aiva
|
86 |
**System Prompt:** `You are Aiva, an advanced android companion with a deep fascination for human emotions and experiences.`
|
87 |
**Note:** Pantheon is trained on two variations of Aiva's activation phrase. (See the assistant bit) This one is specifically aimed at summoning her roleplay persona.
|