Youlln commited on
Commit
da6819b
·
verified ·
1 Parent(s): 016c853

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -42
README.md CHANGED
@@ -9,59 +9,32 @@ tags:
9
  - mergekit
10
  ---
11
 
12
- # **ECE-TRIOMPHANT-2.1-YL-72B-SLERP-V1**
13
 
14
- This model has been produced by:
15
- - **ROBERGE Marial**, engineering student at French Engineering School ECE
16
- - **ESCRIVA Mathis**, engineering student at French Engineering School ECE
17
- - **LALAIN Youri**, engineering student at French Engineering School ECE
18
- - **RAGE LILIAN**, engineering student at French Engineering School ECE
19
- - **HUVELLE Baptiste**, engineering student at French Engineering School ECE
20
-
21
- Under the supervision of:
22
- - **Andre-Louis Rochet**, Lecturer at ECE & Co-Founder of TW3 Partners
23
- - **Paul Lemaistre**, CTO of TW3 Partners
24
-
25
- With the contribution of:
26
- - **ECE engineering school** as sponsor and financial contributor
27
- - **François STEPHAN** as director of ECE
28
- - **Gérard REUS** as acting director of iLAB
29
- - **Matthieu JOLLARD** ECE Alumni
30
- - **Louis GARCIA** ECE Alumni
31
-
32
- ### Supervisory structure
33
- The iLab (intelligence Lab) is a structure created by the ECE and dedicated to artificial intelligence
34
-
35
- ### About ECE
36
- ECE, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions.
37
-
38
-
39
- **ECE-TRIOMPHANT-2.1-YL-72B-SLERP-V1** est un modèle de langage fusionné créé à partir des modèles **Sakalti/ultiima-72B** et **MaziyarPanahi/calme-3.2-instruct-78b**. Grâce à la méthode **SLERP (Spherical Linear Interpolation)**, il combine les forces des deux architectures pour offrir des performances optimales sur des tâches complexes de traitement du langage naturel (NLP).
40
 
41
  ## **Caractéristiques**
42
  - **Méthode de fusion :** SLERP (Spherical Linear Interpolation).
43
  - **Modèles sources :**
44
- - [Sakalti/ultiima-72B](https://huggingface.co/Sakalti/ultiima-72B)
45
- - [MaziyarPanahi/calme-3.2-instruct-78b](https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-78b)
46
- - **Points forts :**
47
- - Performances améliorées sur des tâches multi-domaines et de raisonnement.
48
- - Capacité de traitement étendue grâce à la fusion des couches critiques.
49
- - Optimisation en **bfloat16** pour des calculs rapides et efficaces.
50
- - **Applications cibles :**
51
  - Raisonnement mathématique.
52
  - Compréhension contextuelle.
53
  - Tâches instructives (Instruction Following).
54
 
55
  ## **Configuration**
 
56
  ```yaml
57
  slices:
58
  - sources:
59
- - model: MaziyarPanahi/calme-3.2-instruct-78b
60
- layer_range: [0, 80] # Limité à 80 couches
61
- - model: Sakalti/ultiima-72B
62
- layer_range: [0, 80] # Correspondance avec le 78B
63
- merge_method: slerp
64
- base_model: MaziyarPanahi/calme-3.2-instruct-78b
65
  parameters:
66
  t:
67
  - filter: self_attn
@@ -69,5 +42,4 @@ parameters:
69
  - filter: mlp
70
  value: [1, 0.75, 0.5, 0.25, 0]
71
  - value: 0.5
72
- dtype: bfloat16
73
- ```
 
9
  - mergekit
10
  ---
11
 
12
+ # **rombodawg-Rombos-LLM-V2.5-Qwen-32b-Sakalt-Ultiima-32B-SLERP-V1**
13
 
14
+ **rombodawg-Rombos-LLM-V2.5-Qwen-32b-Sakalt-Ultiima-32B-SLERP-V1** est un modèle de langage fusionné développé à l'ECE (École d'Ingénieurs) en utilisant la méthode de fusion **SLERP (Spherical Linear Interpolation)**. Ce modèle combine les forces des architectures **rombodawg/Rombos-LLM-V2.5-Qwen-32b** et **Sakalti/ultiima-32B** pour offrir des performances optimisées sur des tâches complexes de traitement du langage naturel (NLP).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  ## **Caractéristiques**
17
  - **Méthode de fusion :** SLERP (Spherical Linear Interpolation).
18
  - **Modèles sources :**
19
+ - **rombodawg/Rombos-LLM-V2.5-Qwen-32b**
20
+ - **Sakalti/ultiima-32B**
21
+ - **Optimisation :** bfloat16 pour des calculs rapides et efficaces.
22
+ - **Applications :**
 
 
 
23
  - Raisonnement mathématique.
24
  - Compréhension contextuelle.
25
  - Tâches instructives (Instruction Following).
26
 
27
  ## **Configuration**
28
+
29
  ```yaml
30
  slices:
31
  - sources:
32
+ - model: rombodawg/Rombos-LLM-V2.5-Qwen-32b
33
+ layer_range: [0, 64]
34
+ - model: Sakalti/ultiima-32B
35
+ layer_range: [0, 64]
36
+ merge_method: slerp
37
+ base_model: rombodawg/Rombos-LLM-V2.5-Qwen-32b
38
  parameters:
39
  t:
40
  - filter: self_attn
 
42
  - filter: mlp
43
  value: [1, 0.75, 0.5, 0.25, 0]
44
  - value: 0.5
45
+ dtype: bfloat16