|
--- |
|
license: mit |
|
--- |
|
|
|
# Imran1/QWEN2.5-32B-Translation: Advanced Multilingual Translation Model |
|
|
|
## Overview |
|
**Imran1/QWEN2.5-32B-Translation** is a fine-tuned version of Qwen 2.5 32B, specifically optimized for multilingual translation across **16 different languages**. This model has been extensively fine-tuned to enhance its translation capabilities, making it competitive with high-tier models like 72B in terms of translation accuracy and fluency. |
|
|
|
## Fine-Tuning Process |
|
### Data Collection |
|
To improve the model's understanding and translation capabilities, we curated and synthesized a large dataset consisting of: |
|
- High-quality multilingual conversational datasets. |
|
- Real-world dialogues spanning general, business, and technical domains. |
|
- Translated datasets covering diverse linguistic structures and idiomatic expressions. |
|
|
|
### Multilingual Enhancement |
|
To advance its translation capabilities, we leveraged: |
|
- **Translation Expansion**: The collected dataset was translated into **16 different languages** to ensure robust multilingual performance. |
|
- **Benchmarking Against High-Tier Models**: We utilized state-of-the-art translation models, including **Gemini** and other top-ranking translation models with high BLEU and COMET scores, to refine our translation quality. |
|
- **Reinforcement Learning with Human Feedback (RLHF)**: Translation outputs were evaluated and iteratively improved based on feedback from native speakers and linguistic experts. |
|
|
|
### Training and Optimization |
|
- **Base Model**: Qwen 2.5 32B FP8 |
|
- **Fine-Tuning Framework**: LoRA + QLoRA for efficient training |
|
- **Batch Size**: Optimized for multi-GPU environments |
|
- **Precision**: FP8 for efficient computation without sacrificing performance |
|
- **Training Iterations**: Over 2600 steps on **multi-H100 GPUs** |
|
|
