kyujinpy commited on
Commit
8d731e3
·
1 Parent(s): edde23b

Upload 3 files

Browse files
Files changed (3) hide show
  1. README.md +9 -9
  2. results/copa.jpg +0 -0
  3. results/hella.jpg +0 -0
README.md CHANGED
@@ -9,7 +9,7 @@ pipeline_tag: text-generation
9
  license: cc-by-nc-4.0
10
  ---
11
 
12
- # **Ko-Platypus2-13B**
13
  **More detail repo(Github): [KO-Platypus](https://github.com/Marker-Inc-Korea/KO-Platypus)**
14
  ![KO-Platypus2-13B](./KO_platypus.png)
15
 
@@ -64,7 +64,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
64
 
65
  > Question Answering (QA)
66
  ### COPA (F1)
67
-
68
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
69
  | --- | --- | --- | --- | --- |
70
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.7196 | 0.7193 | 0.7204 | 0.7206 |
@@ -74,11 +74,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
74
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7388 | 0.7626 | 0.7808 | 0.7979 |
75
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
76
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
77
- | **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
78
 
79
  > Natural Language Inference (NLI; 자연어 추론 평가)
80
  ### HellaSwag (F1)
81
-
82
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
83
  | --- | --- | --- | --- | --- |
84
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.5247 | 0.5260 | 0.5278 | 0.5427 |
@@ -88,11 +88,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
88
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4518 | 0.4668 | 0.4726 | 0.4828 |
89
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
90
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
91
- | **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
92
 
93
  > Question Answering (QA)
94
  ### BoolQ (F1)
95
-
96
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
97
  | --- | --- | --- | --- | --- |
98
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.3552 | 0.4751 | 0.4109 | 0.4038 |
@@ -102,11 +102,11 @@ I use A100 GPU 40GB and COLAB, when trianing.
102
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.3607 | 0.6797 | 0.6801 | 0.6622 |
103
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
104
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
105
- | **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
106
 
107
  > Classification
108
  ### SentiNeg (F1)
109
-
110
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
111
  | --- | --- | --- | --- | --- |
112
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.6790 | 0.6257 | 0.5514 | 0.7851 |
@@ -116,7 +116,7 @@ I use A100 GPU 40GB and COLAB, when trianing.
116
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4855 | 0.8295 | 0.8711 | 0.8513 |
117
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
118
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
119
- | **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
120
 
121
  # Implementation Code
122
  ```python
 
9
  license: cc-by-nc-4.0
10
  ---
11
 
12
+ # **Ko-Platypus2-7B-EX**
13
  **More detail repo(Github): [KO-Platypus](https://github.com/Marker-Inc-Korea/KO-Platypus)**
14
  ![KO-Platypus2-13B](./KO_platypus.png)
15
 
 
64
 
65
  > Question Answering (QA)
66
  ### COPA (F1)
67
+ ![jpg](./results/copa.jpg)
68
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
69
  | --- | --- | --- | --- | --- |
70
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.7196 | 0.7193 | 0.7204 | 0.7206 |
 
74
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7388 | 0.7626 | 0.7808 | 0.7979 |
75
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
76
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
77
+ | **KO-platypus2-7B-EX(ours)** | 0.7509 | 0.7899 | 0.8029 | 0.8290 |
78
 
79
  > Natural Language Inference (NLI; 자연어 추론 평가)
80
  ### HellaSwag (F1)
81
+ ![jpg](./results/hella.jpg)
82
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
83
  | --- | --- | --- | --- | --- |
84
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.5247 | 0.5260 | 0.5278 | 0.5427 |
 
88
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4518 | 0.4668 | 0.4726 | 0.4828 |
89
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
90
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
91
+ | **KO-platypus2-7B-EX(ours)** | 0.4571 | 0.4461 | 0.4371 | 0.4525 |
92
 
93
  > Question Answering (QA)
94
  ### BoolQ (F1)
95
+ ![jpg](./results/bool.jpg)
96
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
97
  | --- | --- | --- | --- | --- |
98
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.3552 | 0.4751 | 0.4109 | 0.4038 |
 
102
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.3607 | 0.6797 | 0.6801 | 0.6622 |
103
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
104
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
105
+ | **KO-platypus2-7B-EX(ours)** | 0.6028 | 0.6979 | 0.7016 | NaN |
106
 
107
  > Classification
108
  ### SentiNeg (F1)
109
+ ![jpg](./results/senti.jpg)
110
  | Model | 0-shot | 5-shot | 10-shot | 50-shot |
111
  | --- | --- | --- | --- | --- |
112
  | [Polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) | 0.6790 | 0.6257 | 0.5514 | 0.7851 |
 
116
  | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4855 | 0.8295 | 0.8711 | 0.8513 |
117
  | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
118
  | [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
119
+ | **KO-platypus2-7B-EX(ours)** | 0.5821 | 0.7653 | 0.7991 | NaN |
120
 
121
  # Implementation Code
122
  ```python
results/copa.jpg ADDED
results/hella.jpg ADDED