--- library_name: transformers license: mit --- # RombUltima-32B **FINGU-AI/Ultimos-32B** is a merged model combining **rombodawg/Rombos-LLM-V2.5-Qwen-32b** and **Sakalti/ultiima-32B**. This model maintains the individual strengths of both Qwen and Ultima architectures while benefiting from an optimized fusion for improved reasoning, multilingual comprehension, and multi-turn conversation capabilities. --- ## **Training & Fine-Tuning** RombUltima-32B is based on a **slerp merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima. - **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage. - **Precision:** Trained and fine-tuned in **bfloat16** for efficient inference. - **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints. - **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**. --- ## **Performance & Benchmarks** ### **OpenLLM Leaderboard** 📌 **Coming Soon** – Evaluation against leading LLM benchmarks. ### **MT-Bench** 📌 **Coming Soon** – Multi-turn conversational performance analysis. --- ## **Usage** You can run this model using the following code: ```python import transformers from transformers import AutoTokenizer # Format prompt message = [ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "What is a Large Language Model?"} ] tokenizer = AutoTokenizer.from_pretrained("FINGU-AI/Ultimos-32B") prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False) # Create pipeline pipeline = transformers.pipeline( "text-generation", model="FINGU-AI/Ultimos-32B", tokenizer=tokenizer ) # Generate text sequences = pipeline( prompt, do_sample=True, temperature=0.7, top_p=0.9, num_return_sequences=1, max_length=200, ) print(sequences[0]['generated_text']) ``` --- ## **Merging Details** - **Parent Models:** - 🟢 **rombodawg/Rombos-LLM-V2.5-Qwen-32b** (weight: 0.5) - 🟢 **Sakalti/ultiima-32B** (weight: 0.5) - **Merge Method:** Linear - **Tokenizer Source:** Union-based - **Precision:** Float16 --- ## **Licensing & Intended Use** - **License:** Subject to original licenses of the merged models. - **Intended Use:** Research, content generation, multilingual applications, and general-purpose AI assistance. - **Limitations:** While the model excels in structured reasoning and multilingual understanding, **hallucinations and biases may still exist**. 📌 **For feedback and contributions, visit: [FINGU-AI on Hugging Face](https://huggingface.co/FINGU-AI).**