Upload 3 files
Browse files- README.md +9 -9
- results/copa.jpg +0 -0
- results/hella.jpg +0 -0
README.md
CHANGED
@@ -9,7 +9,7 @@ pipeline_tag: text-generation
|
|
9 |
license: cc-by-nc-4.0
|
10 |
---
|
11 |
|
12 |
-
# **Ko-Platypus2-
|
13 |
**More detail repo(Github): [KO-Platypus](https://github.com/Marker-Inc-Korea/KO-Platypus)**
|
14 |
![KO-Platypus2-13B](./KO_platypus.png)
|
15 |
|
@@ -64,7 +64,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
64 |
|
65 |
> Question Answering (QA)
|
66 |
### COPA (F1)
|
67 |
-
|
68 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
69 |
| --- | --- | --- | --- | --- |
|
70 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.7196 | 0.7193 | 0.7204 | 0.7206 |
|
@@ -74,11 +74,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
74 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7388 | 0.7626 | 0.7808 | 0.7979 |
|
75 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
|
76 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
|
77 |
-
| **KO-platypus2-7B-EX(ours)** |
|
78 |
|
79 |
> Natural Language Inference (NLI; 자연어 추론 평가)
|
80 |
### HellaSwag (F1)
|
81 |
-
|
82 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
83 |
| --- | --- | --- | --- | --- |
|
84 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.5247 | 0.5260 | 0.5278 | 0.5427 |
|
@@ -88,11 +88,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
88 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4518 | 0.4668 | 0.4726 | 0.4828 |
|
89 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
|
90 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
|
91 |
-
| **KO-platypus2-7B-EX(ours)** |
|
92 |
|
93 |
> Question Answering (QA)
|
94 |
### BoolQ (F1)
|
95 |
-
|
96 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
97 |
| --- | --- | --- | --- | --- |
|
98 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.3552 | 0.4751 | 0.4109 | 0.4038 |
|
@@ -102,11 +102,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
102 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.3607 | 0.6797 | 0.6801 | 0.6622 |
|
103 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
|
104 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
|
105 |
-
| **KO-platypus2-7B-EX(ours)** |
|
106 |
|
107 |
> Classification
|
108 |
### SentiNeg (F1)
|
109 |
-
|
110 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
111 |
| --- | --- | --- | --- | --- |
|
112 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.6790 | 0.6257 | 0.5514 | 0.7851 |
|
@@ -116,7 +116,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
|
|
116 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4855 | 0.8295 | 0.8711 | 0.8513 |
|
117 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
|
118 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
|
119 |
-
| **KO-platypus2-7B-EX(ours)** |
|
120 |
|
121 |
# Implementation Code
|
122 |
```python
|
|
|
9 |
license: cc-by-nc-4.0
|
10 |
---
|
11 |
|
12 |
+
# **Ko-Platypus2-7B-EX**
|
13 |
**More detail repo(Github): [KO-Platypus](https://github.com/Marker-Inc-Korea/KO-Platypus)**
|
14 |
![KO-Platypus2-13B](./KO_platypus.png)
|
15 |
|
|
|
64 |
|
65 |
> Question Answering (QA)
|
66 |
### COPA (F1)
|
67 |
+
![jpg](./results/copa.jpg)
|
68 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
69 |
| --- | --- | --- | --- | --- |
|
70 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.7196 | 0.7193 | 0.7204 | 0.7206 |
|
|
|
74 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7388 | 0.7626 | 0.7808 | 0.7979 |
|
75 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
|
76 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
|
77 |
+
| **KO-platypus2-7B-EX(ours)** | 0.7509 | 0.7899 | 0.8029 | 0.8290 |
|
78 |
|
79 |
> Natural Language Inference (NLI; 자연어 추론 평가)
|
80 |
### HellaSwag (F1)
|
81 |
+
![jpg](./results/hella.jpg)
|
82 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
83 |
| --- | --- | --- | --- | --- |
|
84 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.5247 | 0.5260 | 0.5278 | 0.5427 |
|
|
|
88 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4518 | 0.4668 | 0.4726 | 0.4828 |
|
89 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
|
90 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
|
91 |
+
| **KO-platypus2-7B-EX(ours)** | 0.4571 | 0.4461 | 0.4371 | 0.4525 |
|
92 |
|
93 |
> Question Answering (QA)
|
94 |
### BoolQ (F1)
|
95 |
+
![jpg](./results/bool.jpg)
|
96 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
97 |
| --- | --- | --- | --- | --- |
|
98 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.3552 | 0.4751 | 0.4109 | 0.4038 |
|
|
|
102 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.3607 | 0.6797 | 0.6801 | 0.6622 |
|
103 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
|
104 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
|
105 |
+
| **KO-platypus2-7B-EX(ours)** | 0.6028 | 0.6979 | 0.7016 | NaN |
|
106 |
|
107 |
> Classification
|
108 |
### SentiNeg (F1)
|
109 |
+
![jpg](./results/senti.jpg)
|
110 |
| Model | 0-shot | 5-shot | 10-shot | 50-shot |
|
111 |
| --- | --- | --- | --- | --- |
|
112 |
| [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.6790 | 0.6257 | 0.5514 | 0.7851 |
|
|
|
116 |
| [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4855 | 0.8295 | 0.8711 | 0.8513 |
|
117 |
| [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
|
118 |
| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
|
119 |
+
| **KO-platypus2-7B-EX(ours)** | 0.5821 | 0.7653 | 0.7991 | NaN |
|
120 |
|
121 |
# Implementation Code
|
122 |
```python
|
results/copa.jpg
ADDED
![]() |
results/hella.jpg
ADDED
![]() |