Gryphe commited on
Commit
a53877c
·
verified ·
1 Parent(s): 4989595

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -3
README.md CHANGED
@@ -14,6 +14,8 @@ language:
14
  # Pantheon-RP-1.5-12b-Nemo
15
  Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
16
 
 
 
17
  ## Model details
18
  This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first;
19
 
@@ -25,21 +27,22 @@ This time around I went for a multi-stage finetuning process as Mistral Nemo was
25
 
26
  ## Inference
27
 
28
- Despite Mistral's insistence to use lower temperatures I continue to recommend the following settings for inference:
29
  ```
30
- "temperature": 1.0,
31
  "repetition_penalty": 1.05,
32
  "top_p": 0.95
33
  "top_k": 40
34
  "min_p": 0.05
35
  ```
 
36
  Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
37
 
38
  **Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
39
 
40
  ## General Roleplay
41
 
42
- The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style (aka Markdown), as that is the style my Pantheon Roleplay dataset uses.
43
 
44
  There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts.
45
 
 
14
  # Pantheon-RP-1.5-12b-Nemo
15
  Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of personas that can be summoned with a simple activation phrase. The huge variety in personalities introduced also serve to enhance the general roleplay experience.
16
 
17
+ **Disclaimer:** Despite my goal to create the perfect Pantheon finetune I still feel like I've been unable to shave some rougher edges off the Nemo base model. Rather then continue to bash my head against the wall (and as a result not release anything) I've instead decided to release my finest attempt so far as it should already surpass my 1.0 release.
18
+
19
  ## Model details
20
  This time around I went for a multi-stage finetuning process as Mistral Nemo was proving to be somewhat stubborn without a solid base training being performed first;
21
 
 
27
 
28
  ## Inference
29
 
30
+ Nemo is a somewhat strange model when it comes to temperatures so I highly encourage you to experiment to see which works best.
31
  ```
32
+ "temperature": 0.3-1.0,
33
  "repetition_penalty": 1.05,
34
  "top_p": 0.95
35
  "top_k": 40
36
  "min_p": 0.05
37
  ```
38
+
39
  Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
40
 
41
  **Note:** My previous release suffered from a tendency to generate shorter roleplay responses, which I now believe has been mostly resolved.
42
 
43
  ## General Roleplay
44
 
45
+ The second finetune was focused solely on an asterisk-style, no quotes for speech roleplay style (aka Markdown), as that is the style my Pantheon Roleplay dataset uses. I expect there to be a bias inside the model itself towards responding in this style.
46
 
47
  There are no strict rules in regards to character card formatting as the model was trained with a wide variety of inputs, from raw character cards to detailed instructional prompts.
48