Update README.md
Browse files
README.md
CHANGED
@@ -9,10 +9,10 @@ license: mit
|
|
9 |
---
|
10 |
|
11 |
## **Training & Fine-Tuning**
|
12 |
-
RombUltima-32B is based on a **
|
13 |
|
14 |
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
|
15 |
-
- **Precision:** Trained and fine-tuned in **
|
16 |
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
|
17 |
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
|
18 |
|
|
|
9 |
---
|
10 |
|
11 |
## **Training & Fine-Tuning**
|
12 |
+
RombUltima-32B is based on a **slerp merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima.
|
13 |
|
14 |
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
|
15 |
+
- **Precision:** Trained and fine-tuned in **bfloat16** for efficient inference.
|
16 |
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
|
17 |
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
|
18 |
|