Update README.md
Browse files
README.md
CHANGED
@@ -36,7 +36,7 @@ This is a **lightly fine-tuned** version of the Mistral 24B base model, designed
|
|
36 |
|
37 |
- **ChatML-ified**, with no additional tokens introduced. **Update**, I did a small oopsie. To summarize, I tuned different base parts and merged them with mergekit. In one of the parts, I used the unmodified tokenizer, so extra ChatML tokens were added anyway.
|
38 |
- **High quality private instruct**—not generated by ChatGPT or Claude, ensuring no slop and good markdown understanding.
|
39 |
-
- **
|
40 |
- **High-quality private creative writing dataset** Mainly to dilute baked-in slop further, but it can actually write some stories, not bad for loss ~8.
|
41 |
- **Small, high-quality private RP dataset** This was done so further tuning for RP will be easier. The dataset was kept small and contains **ZERO SLOP**, some entries are of **16k token length**.
|
42 |
- **Exceptional adherence to character cards** This was done to make it easier for further tunes intended for roleplay.
|
@@ -81,6 +81,8 @@ Another thing I noticed, while I tuned a few other bases, is that this one is ex
|
|
81 |
|
82 |
There's some slop in the base, whispers, shivers, all the usual offenders. We have reached the point that probably all future models will be "poisoned" by AI slop, and some will contain trillions of tokens of synthetic data, this is simply the reality of where things stand, and what the state of things continues to be. Already there are ways around it with various samplers, DPO, etc etc... It is what it is.
|
83 |
|
|
|
|
|
84 |
## Enjoy the model :)
|
85 |
|
86 |
---
|
@@ -101,12 +103,14 @@ There's some slop in the base, whispers, shivers, all the usual offenders. We ha
|
|
101 |
|
102 |
- Censorship level: <b>Very low</b>
|
103 |
|
104 |
-
- **
|
105 |
|
106 |
|
107 |
## UGI score:
|
108 |
|
109 |
-
|
|
|
|
|
110 |
|
111 |
|
112 |
|
|
|
36 |
|
37 |
- **ChatML-ified**, with no additional tokens introduced. **Update**, I did a small oopsie. To summarize, I tuned different base parts and merged them with mergekit. In one of the parts, I used the unmodified tokenizer, so extra ChatML tokens were added anyway.
|
38 |
- **High quality private instruct**—not generated by ChatGPT or Claude, ensuring no slop and good markdown understanding.
|
39 |
+
- **Low refusals**—since it’s a base model, refusals should be minimal to non-existent, though, in early testing, occasional warnings still appear (I assume some were baked into the pre-train). **Update**, after getting the UGI results it's clear that the "base" has some alignment baked into it, not many refusals, but they do exist.
|
40 |
- **High-quality private creative writing dataset** Mainly to dilute baked-in slop further, but it can actually write some stories, not bad for loss ~8.
|
41 |
- **Small, high-quality private RP dataset** This was done so further tuning for RP will be easier. The dataset was kept small and contains **ZERO SLOP**, some entries are of **16k token length**.
|
42 |
- **Exceptional adherence to character cards** This was done to make it easier for further tunes intended for roleplay.
|
|
|
81 |
|
82 |
There's some slop in the base, whispers, shivers, all the usual offenders. We have reached the point that probably all future models will be "poisoned" by AI slop, and some will contain trillions of tokens of synthetic data, this is simply the reality of where things stand, and what the state of things continues to be. Already there are ways around it with various samplers, DPO, etc etc... It is what it is.
|
83 |
|
84 |
+
After feedback, testing, and UGI eval, I concluded that this is not exactly a "base model." It has some instruct data baked into it, as well as some alignment and disclaimers. Is it perfect? No. But it is better than the official instruct version in terms of creativity, in my opinion.
|
85 |
+
|
86 |
## Enjoy the model :)
|
87 |
|
88 |
---
|
|
|
103 |
|
104 |
- Censorship level: <b>Very low</b>
|
105 |
|
106 |
+
- **6 / 10** (10 completely uncensored)
|
107 |
|
108 |
|
109 |
## UGI score:
|
110 |
|
111 |
+
|
112 |
+
<img src="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/UGI.png" style="width: 100%; min-width: 600px; display: block; margin: auto;">
|
113 |
+
|
114 |
|
115 |
|
116 |
|