Update README.md
Browse files
README.md
CHANGED
@@ -9,59 +9,32 @@ tags:
|
|
9 |
- mergekit
|
10 |
---
|
11 |
|
12 |
-
# **
|
13 |
|
14 |
-
|
15 |
-
- **ROBERGE Marial**, engineering student at French Engineering School ECE
|
16 |
-
- **ESCRIVA Mathis**, engineering student at French Engineering School ECE
|
17 |
-
- **LALAIN Youri**, engineering student at French Engineering School ECE
|
18 |
-
- **RAGE LILIAN**, engineering student at French Engineering School ECE
|
19 |
-
- **HUVELLE Baptiste**, engineering student at French Engineering School ECE
|
20 |
-
|
21 |
-
Under the supervision of:
|
22 |
-
- **Andre-Louis Rochet**, Lecturer at ECE & Co-Founder of TW3 Partners
|
23 |
-
- **Paul Lemaistre**, CTO of TW3 Partners
|
24 |
-
|
25 |
-
With the contribution of:
|
26 |
-
- **ECE engineering school** as sponsor and financial contributor
|
27 |
-
- **François STEPHAN** as director of ECE
|
28 |
-
- **Gérard REUS** as acting director of iLAB
|
29 |
-
- **Matthieu JOLLARD** ECE Alumni
|
30 |
-
- **Louis GARCIA** ECE Alumni
|
31 |
-
|
32 |
-
### Supervisory structure
|
33 |
-
The iLab (intelligence Lab) is a structure created by the ECE and dedicated to artificial intelligence
|
34 |
-
|
35 |
-
### About ECE
|
36 |
-
ECE, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions.
|
37 |
-
|
38 |
-
|
39 |
-
**ECE-TRIOMPHANT-2.1-YL-72B-SLERP-V1** est un modèle de langage fusionné créé à partir des modèles **Sakalti/ultiima-72B** et **MaziyarPanahi/calme-3.2-instruct-78b**. Grâce à la méthode **SLERP (Spherical Linear Interpolation)**, il combine les forces des deux architectures pour offrir des performances optimales sur des tâches complexes de traitement du langage naturel (NLP).
|
40 |
|
41 |
## **Caractéristiques**
|
42 |
- **Méthode de fusion :** SLERP (Spherical Linear Interpolation).
|
43 |
- **Modèles sources :**
|
44 |
-
-
|
45 |
-
-
|
46 |
-
- **
|
47 |
-
|
48 |
-
- Capacité de traitement étendue grâce à la fusion des couches critiques.
|
49 |
-
- Optimisation en **bfloat16** pour des calculs rapides et efficaces.
|
50 |
-
- **Applications cibles :**
|
51 |
- Raisonnement mathématique.
|
52 |
- Compréhension contextuelle.
|
53 |
- Tâches instructives (Instruction Following).
|
54 |
|
55 |
## **Configuration**
|
|
|
56 |
```yaml
|
57 |
slices:
|
58 |
- sources:
|
59 |
-
- model:
|
60 |
-
layer_range: [0,
|
61 |
-
- model: Sakalti/ultiima-
|
62 |
-
layer_range: [0,
|
63 |
-
merge_method: slerp
|
64 |
-
base_model:
|
65 |
parameters:
|
66 |
t:
|
67 |
- filter: self_attn
|
@@ -69,5 +42,4 @@ parameters:
|
|
69 |
- filter: mlp
|
70 |
value: [1, 0.75, 0.5, 0.25, 0]
|
71 |
- value: 0.5
|
72 |
-
dtype: bfloat16
|
73 |
-
```
|
|
|
9 |
- mergekit
|
10 |
---
|
11 |
|
12 |
+
# **rombodawg-Rombos-LLM-V2.5-Qwen-32b-Sakalt-Ultiima-32B-SLERP-V1**
|
13 |
|
14 |
+
**rombodawg-Rombos-LLM-V2.5-Qwen-32b-Sakalt-Ultiima-32B-SLERP-V1** est un modèle de langage fusionné développé à l'ECE (École d'Ingénieurs) en utilisant la méthode de fusion **SLERP (Spherical Linear Interpolation)**. Ce modèle combine les forces des architectures **rombodawg/Rombos-LLM-V2.5-Qwen-32b** et **Sakalti/ultiima-32B** pour offrir des performances optimisées sur des tâches complexes de traitement du langage naturel (NLP).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
## **Caractéristiques**
|
17 |
- **Méthode de fusion :** SLERP (Spherical Linear Interpolation).
|
18 |
- **Modèles sources :**
|
19 |
+
- **rombodawg/Rombos-LLM-V2.5-Qwen-32b**
|
20 |
+
- **Sakalti/ultiima-32B**
|
21 |
+
- **Optimisation :** bfloat16 pour des calculs rapides et efficaces.
|
22 |
+
- **Applications :**
|
|
|
|
|
|
|
23 |
- Raisonnement mathématique.
|
24 |
- Compréhension contextuelle.
|
25 |
- Tâches instructives (Instruction Following).
|
26 |
|
27 |
## **Configuration**
|
28 |
+
|
29 |
```yaml
|
30 |
slices:
|
31 |
- sources:
|
32 |
+
- model: rombodawg/Rombos-LLM-V2.5-Qwen-32b
|
33 |
+
layer_range: [0, 64]
|
34 |
+
- model: Sakalti/ultiima-32B
|
35 |
+
layer_range: [0, 64]
|
36 |
+
merge_method: slerp
|
37 |
+
base_model: rombodawg/Rombos-LLM-V2.5-Qwen-32b
|
38 |
parameters:
|
39 |
t:
|
40 |
- filter: self_attn
|
|
|
42 |
- filter: mlp
|
43 |
value: [1, 0.75, 0.5, 0.25, 0]
|
44 |
- value: 0.5
|
45 |
+
dtype: bfloat16
|
|