Update README.md
Browse files
README.md
CHANGED
@@ -2,4 +2,54 @@
|
|
2 |
license: other
|
3 |
license_name: freeuse
|
4 |
license_link: LICENSE
|
|
|
|
|
5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
license: other
|
3 |
license_name: freeuse
|
4 |
license_link: LICENSE
|
5 |
+
tags:
|
6 |
+
- not-for-all-audiences
|
7 |
---
|
8 |
+
# Moistral 11B v1 💦
|
9 |
+
|
10 |
+
*An eRP model with a rich vocabulary of moist. Finetuned by yours truly.*
|
11 |
+
|
12 |
+
## Description
|
13 |
+
|
14 |
+
Moistral 11B is an *aggressive* **finetune** of the legendary Fimbulvert v2 model.
|
15 |
+
|
16 |
+
If you haven't heard of that model, then your poor-GPU ass is missing out!
|
17 |
+
It is an amazing substitute for those heavyweight models, and it won't disappoint.
|
18 |
+
(That's coming from someone who daily drives a Venus / Goliath 120B)
|
19 |
+
|
20 |
+
...and if you're looking for something *more moist*, then you've come to the right place.
|
21 |
+
|
22 |
+
You can expect the same sort of performance here! It's the Honda Civic of eRP models!
|
23 |
+
|
24 |
+
### Pimp my ride
|
25 |
+
Turn any old story into a **Moist**ral masterpiece.
|
26 |
+
In fact, we recommend that you leave out any hints of *moist* in your story to _let Moistral cook_.
|
27 |
+
|
28 |
+
### Drive it like it's hot
|
29 |
+
When in Instruct Mode, Moistral will treat you like a director of your own fantasy ride.
|
30 |
+
Be a controlling navigator or a passenger princess, it won't matter - you're in for a good ride!
|
31 |
+
|
32 |
+
### It's got the mileage
|
33 |
+
Moistral has been trained with many long form texts, a nice chunk of which are 8K in length.
|
34 |
+
It is capable of going far and long without passing it back to you. This is not your typical chibi RP model.
|
35 |
+
|
36 |
+
## What's next?
|
37 |
+
Moistral 11B is my first attempt at finetuning a capable model (Sorry, CreamPhi-2).
|
38 |
+
It's coherent and creative enough to let me understand the impact of my dataset & training.
|
39 |
+
Playing around with it has already given me a better idea on the Do's and Don'ts.
|
40 |
+
I will most likely make a version 2 with some improvements:
|
41 |
+
1. Remove any glitchy texts that come from my dataset. Sanitize sanitize sanitize!
|
42 |
+
2. Balance out the themes in the dataset for a richer, more diverse experience.
|
43 |
+
3. Consider extending the context window.
|
44 |
+
4. Add a 'monologue' dataset that forces the model to keep talking without much interaction from the `user`.
|
45 |
+
5. Maybe, just maybe, expose it to dry stuff to let Moistral cook.
|
46 |
+
|
47 |
+
I will not deny that I'm standing on the shoulders of giants.
|
48 |
+
Thank you Sao for sharing your finetune config along with tips on getting started.
|
49 |
+
Thanks to everyone in the Finetuning channel for entertaining my every question.
|
50 |
+
|
51 |
+
---
|
52 |
+
license: other
|
53 |
+
license_name: freeuse
|
54 |
+
license_link: LICENSE
|
55 |
+
---
|