Update README.md
Browse files
README.md
CHANGED
@@ -24,13 +24,37 @@ pipeline_tag: text-generation
|
|
24 |
---
|
25 |
|
26 |
## 13B-Ouroboros
|
27 |
-
Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
|
33 |
-
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
## Composition:
|
36 |
Ouroboros is comprised of 40 layers [LLaMAv1 13B standard] mixed at optimized
|
|
|
24 |
---
|
25 |
|
26 |
## 13B-Ouroboros
|
27 |
+
Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model with a merging technique optimized per layer---
|
28 |
+
tags:
|
29 |
+
- llama
|
30 |
+
- alpaca
|
31 |
+
- vicuna
|
32 |
+
- uncensored
|
33 |
+
- merge
|
34 |
+
- mix
|
35 |
+
- airoboros
|
36 |
+
- openorca
|
37 |
+
- orcamini
|
38 |
+
- orca
|
39 |
+
- instruct
|
40 |
+
- mixtune
|
41 |
+
datasets:
|
42 |
+
- Open-Orca/OpenOrca
|
43 |
+
- anon8231489123/ShareGPT_Vicuna_unfiltered
|
44 |
+
- jondurbin/airoboros-uncensored
|
45 |
+
language:
|
46 |
+
- en
|
47 |
+
metrics:
|
48 |
+
- accuracy
|
49 |
+
pipeline_tag: text-generation
|
50 |
+
---
|
51 |
|
52 |
+
## 13B-Ouroboros
|
53 |
+
Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model using a custom merging technique, tweaking
|
54 |
+
each layer's merge % based on internal tests against the PTB dataset, scoring ~26.31 according to internal evaluation
|
55 |
+
(6 samples, sequence length 1024; this testing is not empirical, it's a quick way to find near-optimum values). Testing,
|
56 |
+
evaluating, and remixing this model is absolutely permissible and even encouraged (within the bounds of Meta's LLaMAv1
|
57 |
+
license agreement); the more feedback the better we can tune our process! 😊
|
58 |
|
59 |
## Composition:
|
60 |
Ouroboros is comprised of 40 layers [LLaMAv1 13B standard] mixed at optimized
|