
# LASSL roberta-ko-small
## How to use

```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("lassl/roberta-ko-small")
tokenizer = AutoTokenizer.from_pretrained("lassl/roberta-ko-small")
```

## Evaluation
Pretrained `roberta-ko-small` on korean language was trained by [LASSL](https://github.com/lassl/lassl) framework. Below performance was evaluated at 2021/12/15.

| nsmc | klue_nli | klue_sts | korquadv1 | klue_mrc | avg |
| ---- | -------- | -------- | --------- | ---- | -------- |
| 87.8846 | 66.3086 | 83.8353 | 83.1780 | 42.4585 | 72.7330 |

## Corpora
This model was trained from 6,860,062 examples (whose have 3,512,351,744 tokens). 6,860,062 examples are extracted from below corpora. If you want to get information for training, you should see `config.json`.  

```bash
corpora/
├── [707M]  kowiki_latest.txt
├── [ 26M]  modu_dialogue_v1.2.txt
├── [1.3G]  modu_news_v1.1.txt
├── [9.7G]  modu_news_v2.0.txt
├── [ 15M]  modu_np_v1.1.txt
├── [1008M]  modu_spoken_v1.2.txt
├── [6.5G]  modu_written_v1.0.txt
└── [413M]  petition.txt
```




{"language": "ko", "license": "apache-2.0", "tags": ["korean", "lassl"], "mask_token": "<mask>", "widget": [{"text": "\ub300\ud55c\ubbfc\uad6d\uc758 \uc218\ub3c4\ub294 <mask> \uc785\ub2c8\ub2e4."}]}
transformers pytorch roberta fill-mask korean lassl ko license:apache-2.0 autotrain_compatible endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #roberta #fill-mask #korean #lassl #ko #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 

LASSL roberta-ko-small
======================


How to use
----------


Evaluation
----------


Pretrained 'roberta-ko-small' on korean language was trained by LASSL framework. Below performance was evaluated at 2021/12/15.



Corpora
-------


This model was trained from 6,860,062 examples (whose have 3,512,351,744 tokens). 6,860,062 examples are extracted from below corpora. If you want to get information for training, you should see 'URL'.



TAGS
#transformers #pytorch #roberta #fill-mask #korean #lassl #ko #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 

passage: TAGS
#transformers #pytorch #roberta #fill-mask #korean #lassl #ko #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 
