FINGU-AI commited on
Commit
7bd9c64
·
verified ·
1 Parent(s): ad5d7e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -9,10 +9,10 @@ license: mit
9
  ---
10
 
11
  ## **Training & Fine-Tuning**
12
- RombUltima-32B is based on a **linear merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima.
13
 
14
  - **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
15
- - **Precision:** Trained and fine-tuned in **float16** for efficient inference.
16
  - **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
17
  - **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
18
 
 
9
  ---
10
 
11
  ## **Training & Fine-Tuning**
12
+ RombUltima-32B is based on a **slerp merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima.
13
 
14
  - **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
15
+ - **Precision:** Trained and fine-tuned in **bfloat16** for efficient inference.
16
  - **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
17
  - **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
18