Tarek07 commited on
Commit
459109d
·
verified ·
1 Parent(s): e499b6b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +63 -62
README.md CHANGED
@@ -1,62 +1,63 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Progenitor-V3.2-70B
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using D:/mergekit_models/L3.3-70B-Euryale-v2.3 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * D:/mergekit_models/Negative_LLAMA_70B
22
- * D:/mergekit_models/L3.1-70B-Hanami-x1
23
- * D:/mergekit_models/EVA-LLaMA-3.33-70B-v0.1
24
- * D:/mergekit_models//70B-L3.3-Cirrus-x1
25
- * D:/mergekit_models/Anubis-70B-v1
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- models:
33
- - model: D:/mergekit_models/L3.1-70B-Hanami-x1
34
- parameters:
35
- weight: 0.20
36
- density: 0.7
37
- - model: D:/mergekit_models//70B-L3.3-Cirrus-x1
38
- parameters:
39
- weight: 0.20
40
- density: 0.7
41
- - model: D:/mergekit_models/Negative_LLAMA_70B
42
- parameters:
43
- weight: 0.20
44
- density: 0.7
45
- - model: D:/mergekit_models/Anubis-70B-v1
46
- parameters:
47
- weight: 0.20
48
- density: 0.7
49
- - model: D:/mergekit_models/EVA-LLaMA-3.33-70B-v0.1
50
- parameters:
51
- weight: 0.20
52
- density: 0.7
53
- merge_method: della_linear
54
- base_model: D:/mergekit_models/L3.3-70B-Euryale-v2.3
55
- parameters:
56
- epsilon: 0.2
57
- lambda: 1.1
58
- dype: float32
59
- out_dtype: bfloat16
60
- tokenizer:
61
- source: base
62
- ```
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+ license: llama3.3
8
+ ---
9
+ I was curious about using Euryale as the base for Progenitor for a while, this is the result.
10
+ # Progenitor-V3.2-70B
11
+
12
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
+
14
+ ## Merge Details
15
+ ### Merge Method
16
+
17
+ This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using D:/mergekit_models/L3.3-70B-Euryale-v2.3 as a base.
18
+
19
+ ### Models Merged
20
+
21
+ The following models were included in the merge:
22
+ * D:/mergekit_models/Negative_LLAMA_70B
23
+ * D:/mergekit_models/L3.1-70B-Hanami-x1
24
+ * D:/mergekit_models/EVA-LLaMA-3.33-70B-v0.1
25
+ * D:/mergekit_models//70B-L3.3-Cirrus-x1
26
+ * D:/mergekit_models/Anubis-70B-v1
27
+
28
+ ### Configuration
29
+
30
+ The following YAML configuration was used to produce this model:
31
+
32
+ ```yaml
33
+ models:
34
+ - model: D:/mergekit_models/L3.1-70B-Hanami-x1
35
+ parameters:
36
+ weight: 0.20
37
+ density: 0.7
38
+ - model: D:/mergekit_models//70B-L3.3-Cirrus-x1
39
+ parameters:
40
+ weight: 0.20
41
+ density: 0.7
42
+ - model: D:/mergekit_models/Negative_LLAMA_70B
43
+ parameters:
44
+ weight: 0.20
45
+ density: 0.7
46
+ - model: D:/mergekit_models/Anubis-70B-v1
47
+ parameters:
48
+ weight: 0.20
49
+ density: 0.7
50
+ - model: D:/mergekit_models/EVA-LLaMA-3.33-70B-v0.1
51
+ parameters:
52
+ weight: 0.20
53
+ density: 0.7
54
+ merge_method: della_linear
55
+ base_model: D:/mergekit_models/L3.3-70B-Euryale-v2.3
56
+ parameters:
57
+ epsilon: 0.2
58
+ lambda: 1.1
59
+ dype: float32
60
+ out_dtype: bfloat16
61
+ tokenizer:
62
+ source: base
63
+ ```