

This is RoBERTa model pretrained on texts in the Japanese language.

3.45GB wikipedia text

 trained 125M step
 
  use the BERT BPE tokenizer.
  
  If you want to fine-tune model. Please use 
```python


from transformers import BertTokenizer, RoFormerModel
BertTokenizer.from_pretrained('Roformer-base-japanese')
RoFormerModel.from_pretrained('Roformer-base-japanese')
```
 
  
  The accuracy in JGLUE-marc-ja-v1.0 binary sentiment classification 95.12%

  
  
  
 Contribute by Yokohama Nationaly University Mori Lab
 
 @article{su2021roformer,
  title={Roformer: Enhanced transformer with rotary position embedding},
  author={Su, Jianlin and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng},
  journal={arXiv preprint arXiv:2104.09864},
  year={2021}
}
{"license": "apache-2.0", "widget": [{"text": "\u6a2a\u6d5c\u56fd\u7acb\u5927\u5b66\u306f\u65e5\u672c\u306e[MASK]\u5948\u5ddd\u770c\u306b\u3042\u308b\u3002"}]}
transformers pytorch safetensors roberta fill-mask license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #safetensors #roberta #fill-mask #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 



This is RoBERTa model pretrained on texts in the Japanese language.

3.45GB wikipedia text

 trained 125M step
 
  use the BERT BPE tokenizer.
  
  If you want to fine-tune model. Please use 

 
  
  The accuracy in JGLUE-marc-ja-v1.0 binary sentiment classification 95.12%

  
  
  
 Contribute by Yokohama Nationaly University Mori Lab
 
 @article{su2021roformer,
  title={Roformer: Enhanced transformer with rotary position embedding},
  author={Su, Jianlin and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng},
  journal={arXiv preprint arXiv:2104.09864},
  year={2021}
}
TAGS
#transformers #pytorch #safetensors #roberta #fill-mask #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #safetensors #roberta #fill-mask #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
