digitous commited on
Commit
f77e039
·
1 Parent(s): de615ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -6
README.md CHANGED
@@ -24,13 +24,37 @@ pipeline_tag: text-generation
24
  ---
25
 
26
  ## 13B-Ouroboros
27
- Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model using a custom merging script that optimizes
28
- per-layer merging based on a given dataset. Ouroboros is optimized against the PTB text only validation dataset, scoring
29
- ~26.31 according to internal evaluation (6 samples, sequence length 1024; this testing is not empirical, it's a part of
30
- the random search algorithm). Testing, evaluating, and remixing this model is absolutely permissible and even encouraged
31
- (within the bounds of Meta's LLaMAv1 license agreement); the more feedback the better we can tune our process! 😊
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
- When the mix tuning system has reached a certain point of maturity it will be released open source.
 
 
 
 
 
34
 
35
  ## Composition:
36
  Ouroboros is comprised of 40 layers [LLaMAv1 13B standard] mixed at optimized
 
24
  ---
25
 
26
  ## 13B-Ouroboros
27
+ Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model with a merging technique optimized per layer---
28
+ tags:
29
+ - llama
30
+ - alpaca
31
+ - vicuna
32
+ - uncensored
33
+ - merge
34
+ - mix
35
+ - airoboros
36
+ - openorca
37
+ - orcamini
38
+ - orca
39
+ - instruct
40
+ - mixtune
41
+ datasets:
42
+ - Open-Orca/OpenOrca
43
+ - anon8231489123/ShareGPT_Vicuna_unfiltered
44
+ - jondurbin/airoboros-uncensored
45
+ language:
46
+ - en
47
+ metrics:
48
+ - accuracy
49
+ pipeline_tag: text-generation
50
+ ---
51
 
52
+ ## 13B-Ouroboros
53
+ Ouroboros is an experimental model based on Meta's LLaMA [v1] 13B base model using a custom merging technique, tweaking
54
+ each layer's merge % based on internal tests against the PTB dataset, scoring ~26.31 according to internal evaluation
55
+ (6 samples, sequence length 1024; this testing is not empirical, it's a quick way to find near-optimum values). Testing,
56
+ evaluating, and remixing this model is absolutely permissible and even encouraged (within the bounds of Meta's LLaMAv1
57
+ license agreement); the more feedback the better we can tune our process! 😊
58
 
59
  ## Composition:
60
  Ouroboros is comprised of 40 layers [LLaMAv1 13B standard] mixed at optimized