
### Model description
- This model was trained on **ZH, JA, KO**'s Wikipedia (5 epochs).

### How to use
```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("conan1024hao/cjkbert-small")
model = AutoModelForMaskedLM.from_pretrained("conan1024hao/cjkbert-small")
```
- Before you fine-tune downstream tasks, you don't need any text segmentation.
- (Though you may obtain better results if you applied morphological analysis to the data before fine-tuning)

### Morphological analysis tools
- ZH: For Chinese, we use [LTP](https://github.com/HIT-SCIR/ltp).
- JA: For Japanese, we use [Juman++](https://github.com/ku-nlp/jumanpp).
- KO: For Korean, we use [KoNLPy](https://github.com/konlpy/konlpy)(Kkma class).

### Tokenization
- We use character-based tokenization with **whole-word-masking** strategy.

### Model size
- vocab_size: 15015
- num_hidden_layers: 4
- hidden_size: 512
- num_attention_heads: 8
- param_num: 25M
{"language": ["ja", "zh", "ko"], "license": "cc-by-sa-4.0", "datasets": ["wikipedia"], "mask_token": "[MASK]", "widget": [{"text": "\u65e9\u7a32\u7530\u5927\u5b66\u3067\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u3092[MASK]\u3076\u3002"}, {"text": "\u674e\u767d\u662f[MASK]\u671d\u4eba\u3002"}, {"text": "\ubd88\uace0\uae30[MASK] \uba39\uaca0\uc2b5\ub2c8\ub2e4."}]}
transformers pytorch bert fill-mask ja zh ko dataset:wikipedia license:cc-by-sa-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #ja #zh #ko #dataset-wikipedia #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 


### Model description
- This model was trained on ZH, JA, KO's Wikipedia (5 epochs).

### How to use

- Before you fine-tune downstream tasks, you don't need any text segmentation.
- (Though you may obtain better results if you applied morphological analysis to the data before fine-tuning)

### Morphological analysis tools
- ZH: For Chinese, we use LTP.
- JA: For Japanese, we use Juman++.
- KO: For Korean, we use KoNLPy(Kkma class).

### Tokenization
- We use character-based tokenization with whole-word-masking strategy.

### Model size
- vocab_size: 15015
- num_hidden_layers: 4
- hidden_size: 512
- num_attention_heads: 8
- param_num: 25M
### Model description
- This model was trained on ZH, JA, KO's Wikipedia (5 epochs). ### How to use

- Before you fine-tune downstream tasks, you don't need any text segmentation.
- (Though you may obtain better results if you applied morphological analysis to the data before fine-tuning) ### Morphological analysis tools
- ZH: For Chinese, we use LTP.
- JA: For Japanese, we use Juman++.
- KO: For Korean, we use KoNLPy(Kkma class). ### Tokenization
- We use character-based tokenization with whole-word-masking strategy. ### Model size
- vocab_size: 15015
- num_hidden_layers: 4
- hidden_size: 512
- num_attention_heads: 8
- param_num: 25M
TAGS
#transformers #pytorch #bert #fill-mask #ja #zh #ko #dataset-wikipedia #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Model description
- This model was trained on ZH, JA, KO's Wikipedia (5 epochs). ### How to use

- Before you fine-tune downstream tasks, you don't need any text segmentation.
- (Though you may obtain better results if you applied morphological analysis to the data before fine-tuning) ### Morphological analysis tools
- ZH: For Chinese, we use LTP.
- JA: For Japanese, we use Juman++.
- KO: For Korean, we use KoNLPy(Kkma class). ### Tokenization
- We use character-based tokenization with whole-word-masking strategy. ### Model size
- vocab_size: 15015
- num_hidden_layers: 4
- hidden_size: 512
- num_attention_heads: 8
- param_num: 25M
passage: TAGS
#transformers #pytorch #bert #fill-mask #ja #zh #ko #dataset-wikipedia #license-cc-by-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
### Model description
- This model was trained on ZH, JA, KO's Wikipedia (5 epochs).### How to use

- Before you fine-tune downstream tasks, you don't need any text segmentation.
- (Though you may obtain better results if you applied morphological analysis to the data before fine-tuning)### Morphological analysis tools
- ZH: For Chinese, we use LTP.
- JA: For Japanese, we use Juman++.
- KO: For Korean, we use KoNLPy(Kkma class).### Tokenization
- We use character-based tokenization with whole-word-masking strategy.### Model size
- vocab_size: 15015
- num_hidden_layers: 4
- hidden_size: 512
- num_attention_heads: 8
- param_num: 25M