Update README.md
Browse files
README.md
CHANGED
|
@@ -24,18 +24,18 @@ Mify-Coder-2.5B-v1 is a **2.5B-parameter code-focused language model**. It deliv
|
|
| 24 |
|
| 25 |
## **Performance Highlights**
|
| 26 |
|
| 27 |
-
| **Category** | **Benchmark**
|
| 28 |
-
|
| 29 |
-
| Code Gen | MBPP
|
| 30 |
-
| Code Gen | MBPP+
|
| 31 |
-
| Code Gen | HumanEval
|
| 32 |
-
| Code Gen | HumanEval+
|
| 33 |
-
| Code Gen | NumpyEval
|
| 34 |
-
| Code Gen | PandasEval
|
| 35 |
-
| Tool Use | BFCL v1
|
| 36 |
-
| Tool Use | BFCL v2
|
| 37 |
-
| Safety | AIR-Bench
|
| 38 |
-
| SecCode Gen | CybersecEval4-Autocomplete
|
| 39 |
|
| 40 |
|
| 41 |
- Outperforms larger models on algorithmic reasoning tasks while maintaining competitive general coding and security-oriented capabilities.
|
|
|
|
| 24 |
|
| 25 |
## **Performance Highlights**
|
| 26 |
|
| 27 |
+
| **Category** | **Benchmark** | **# Shots** | **Metric** | **Scores** |
|
| 28 |
+
|----------------|--------------------------------------|-------------|------------|--------------|
|
| 29 |
+
| Code Gen | MBPP | 0 | pass@1 | 91.21% |
|
| 30 |
+
| Code Gen | MBPP+ | 0 | pass@1 | 89.15% |
|
| 31 |
+
| Code Gen | HumanEval | 0 | pass@1 | 53.66% |
|
| 32 |
+
| Code Gen | HumanEval+ | 0 | pass@1 | 48.78% |
|
| 33 |
+
| Code Gen | NumpyEval | 0 | pass@1 | 56.44% |
|
| 34 |
+
| Code Gen | PandasEval | 0 | pass@1 | 53.47% |
|
| 35 |
+
| Tool Use | BFCL v1 | 0 | acc | 79.19% |
|
| 36 |
+
| Tool Use | BFCL v2 | 0 | acc | 55.26% |
|
| 37 |
+
| Safety | AIR-Bench | 0 | pass@1 | 67.32% |
|
| 38 |
+
| SecCode Gen | CybersecEval4-Autocomplete | 0 | pass@1 | 78.91% |
|
| 39 |
|
| 40 |
|
| 41 |
- Outperforms larger models on algorithmic reasoning tasks while maintaining competitive general coding and security-oriented capabilities.
|