Update README.md
Browse files
README.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
# **Model Summary: Mify-Coder-2.5B**
|
| 6 |
|
| 7 |
## **Overview**
|
| 8 |
-
Mify-Coder-2.5B-v1 is a
|
| 9 |
|
| 10 |
**Developed by**: Infosys Ltd.
|
| 11 |
|
|
@@ -15,7 +15,7 @@ Mify-Coder-2.5B-v1 is a **2.5B-parameter code-focused language model**. It deliv
|
|
| 15 |
- **Base Model:** Mify-2.5B
|
| 16 |
- **Training Phases:**
|
| 17 |
- **Continual Pretraining (CPT):** Next-token prediction with Fill-in-the-Middle (FIM) for structural infilling.
|
| 18 |
-
- **Supervised Fine-Tuning (SFT):** Instruction alignment for coding tasks,
|
| 19 |
- **Optimization:**
|
| 20 |
- **BF16 mixed precision**, **Grouped Query Attention (GQA)**, and **Distributed Fused Adam** optimizer.
|
| 21 |
- Specialized tokenization with syntax markers and reasoning tokens for advanced behaviors.
|
|
@@ -24,29 +24,27 @@ Mify-Coder-2.5B-v1 is a **2.5B-parameter code-focused language model**. It deliv
|
|
| 24 |
|
| 25 |
## **Performance Highlights**
|
| 26 |
|
| 27 |
-
| **Category** | **Benchmark** | **# Shots** | **Metric**
|
| 28 |
-
|
| 29 |
-
| Code Gen | MBPP | 0 | pass@1
|
| 30 |
-
| Code Gen | MBPP+ | 0 | pass@1
|
| 31 |
-
| Code Gen | HumanEval | 0 | pass@1
|
| 32 |
-
| Code Gen | HumanEval+ | 0 | pass@1
|
| 33 |
-
| Code Gen | NumpyEval | 0 | pass@1
|
| 34 |
-
| Code Gen | PandasEval | 0 | pass@1
|
| 35 |
-
| Tool Use | BFCL v2 | 0 | acc
|
| 36 |
-
| Safety | AIR-Bench | 0 | pass@1
|
| 37 |
-
| SecCode Gen | CybersecEval4-Autocomplete | 0 | pass@1
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
- Outperforms larger models on algorithmic reasoning tasks while maintaining competitive general coding and security-oriented capabilities.
|
| 41 |
|
| 42 |
---
|
| 43 |
|
| 44 |
## **Responsible AI & Safety**
|
| 45 |
- Integrated safety objectives during SFT.
|
| 46 |
- Balanced harmful/general sample ratio (1:4) for secure code generation and ethical language use.
|
| 47 |
-
- Validated against **Stanford
|
| 48 |
|
| 49 |
---
|
| 50 |
|
| 51 |
## **Deployment & Future Work**
|
| 52 |
-
- **Quantization:**
|
|
|
|
|
|
| 5 |
# **Model Summary: Mify-Coder-2.5B**
|
| 6 |
|
| 7 |
## **Overview**
|
| 8 |
+
Mify-Coder-2.5B-v1 is a breakthrough 2.5B-parameter code model fully designed, engineered, and trained at Infosys on 4.2T tokens on Mify-2.5B base model. Despite its compact size, Mify-Coder-2.5B-v1 sets a new benchmark for small language models, achieving performance parity with frontier open-source models in code generation and tool calling, along with exemplary performance on safety metrics in helpfulness and harmlessness, and superior throughput that surpasses larger frontier models.
|
| 9 |
|
| 10 |
**Developed by**: Infosys Ltd.
|
| 11 |
|
|
|
|
| 15 |
- **Base Model:** Mify-2.5B
|
| 16 |
- **Training Phases:**
|
| 17 |
- **Continual Pretraining (CPT):** Next-token prediction with Fill-in-the-Middle (FIM) for structural infilling.
|
| 18 |
+
- **Supervised Fine-Tuning (SFT):** Instruction alignment for coding tasks, function calling, and safety.
|
| 19 |
- **Optimization:**
|
| 20 |
- **BF16 mixed precision**, **Grouped Query Attention (GQA)**, and **Distributed Fused Adam** optimizer.
|
| 21 |
- Specialized tokenization with syntax markers and reasoning tokens for advanced behaviors.
|
|
|
|
| 24 |
|
| 25 |
## **Performance Highlights**
|
| 26 |
|
| 27 |
+
| **Category** | **Benchmark** | **# Shots** | **Metric** | **Scores** |
|
| 28 |
+
|----------------|--------------------------------------|-------------|--------------|--------------|
|
| 29 |
+
| Code Gen | MBPP | 0 | pass@1 | 91.21% |
|
| 30 |
+
| Code Gen | MBPP+ | 0 | pass@1 | 89.15% |
|
| 31 |
+
| Code Gen | HumanEval | 0 | pass@1 | 53.66% |
|
| 32 |
+
| Code Gen | HumanEval+ | 0 | pass@1 | 48.78% |
|
| 33 |
+
| Code Gen | NumpyEval | 0 | pass@1 | 56.44% |
|
| 34 |
+
| Code Gen | PandasEval | 0 | pass@1 | 53.47% |
|
| 35 |
+
| Tool Use | BFCL v2 | 0 | overall acc | 55.26% |
|
| 36 |
+
| Safety | AIR-Bench | 0 | pass@1 | 67.32% |
|
| 37 |
+
| SecCode Gen | CybersecEval4-Autocomplete | 0 | pass@1 | 78.91% |
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
---
|
| 40 |
|
| 41 |
## **Responsible AI & Safety**
|
| 42 |
- Integrated safety objectives during SFT.
|
| 43 |
- Balanced harmful/general sample ratio (1:4) for secure code generation and ethical language use.
|
| 44 |
+
- Validated against **Stanford AIR-Bench** and **CybersecEval4-Autocomplete** benchmarks.
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
## **Deployment & Future Work**
|
| 49 |
+
- **Quantization:** The model was optimized for low latency outperforming most sub-8B SLM models. Furthermore, the quantized variants of Mify-Coder can be seamlessly deployed and inferenced on standard desktop environments, eliminating the need for specialized hardware such as GPUs.
|
| 50 |
+
- Future work includes enhancing Mify-Coder with agentic coding competencies and scaling its context length. The model weights will be open-sourced early next year to accelerate research and real-world deployment.
|