
## Indonesian RoBERTa Base IndoNLI

Indonesian RoBERTa Base IndoNLI is a natural language inference (NLI) model based on the [RoBERTa](https://arxiv.org/abs/1907.11692) model. The model was originally the pre-trained [Indonesian RoBERTa Base](https://hf.co/flax-community/indonesian-roberta-base) model, which is then fine-tuned on [`IndoNLI`](https://github.com/ir-nlp-csui/indonli)'s dataset consisting of Indonesian Wikipedia, news, and Web articles [1].

After training, the model achieved an evaluation/dev accuracy of 77.06%. On the benchmark `test_lay` subset, the model achieved an accuracy of 74.24% and on the benchmark `test_expert` subset, the model achieved an accuracy of 61.66%.

Hugging Face's `Trainer` class from the [Transformers](https://huggingface.co/transformers) library was used to train the model. PyTorch was used as the backend framework during training, but the model remains compatible with other frameworks nonetheless.

## Model

| Model                             | #params | Arch.        | Training/Validation data (text) |
| --------------------------------- | ------- | ------------ | ------------------------------- |
| `indonesian-roberta-base-indonli` | 124M    | RoBERTa Base | `IndoNLI`                       |

## Evaluation Results

The model was trained for 5 epochs, with a batch size of 16, a learning rate of 2e-5, a weight decay of 0.1, and a warmup ratio of 0.2, with linear annealing to 0. The best model was loaded at the end.

| Epoch | Training Loss | Validation Loss | Accuracy |
| ----- | ------------- | --------------- | -------- |
| 1     | 0.989200      | 0.691663        | 0.731452 |
| 2     | 0.673000      | 0.621913        | 0.766045 |
| 3     | 0.449900      | 0.662543        | 0.770596 |
| 4     | 0.293600      | 0.777059        | 0.768320 |
| 5     | 0.194200      | 0.948068        | 0.764224 |

## How to Use

### As NLI Classifier

```python
from transformers import pipeline

pretrained_name = "w11wo/indonesian-roberta-base-indonli"

nlp = pipeline(
    "sentiment-analysis",
    model=pretrained_name,
    tokenizer=pretrained_name
)

nlp("Andi tersenyum karena mendapat hasil baik. </s></s> Andi sedih.")
```

## Disclaimer

Do consider the biases which come from both the pre-trained RoBERTa model and the `IndoNLI` dataset that may be carried over into the results of this model.

## References

[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). [IndoNLI: A Natural Language Inference Dataset for Indonesian](https://arxiv.org/abs/2110.14566). _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.

## Author

Indonesian RoBERTa Base IndoNLI was trained and evaluated by [Wilson Wongso](https://w11wo.github.io/). All computation and development are done on Google Colaboratory using their free GPU access.

{"language": "id", "license": "mit", "tags": ["indonesian-roberta-base-indonli"], "datasets": ["indonli"], "widget": [{"text": "Andi tersenyum karena mendapat hasil baik. </s></s> Andi sedih."}], "model-index": [{"name": "w11wo/indonesian-roberta-base-indonli", "results": [{"task": {"type": "natural-language-inference", "name": "Natural Language Inference"}, "dataset": {"name": "indonli", "type": "indonli", "config": "indonli", "split": "test_expert"}, "metrics": [{"type": "accuracy", "value": 0.6072386058981233, "name": "Accuracy", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzY4NDkwNTdlYjI4MzY3ZTk3NmZjYjA1MjE2YWQ5MjJjMGM3NTc1NWVjODQzNTc1ZTYyZWVmYmY5NTI3NWY1ZSIsInZlcnNpb24iOjF9.Aeo_Id90j2JtyApv3LvJHkQtHz-9wO4SNvTdb8O_pp0KFQGfWXnkgX2t2hafIUxSKmZbETIte-FaPbZ9AGZSDA"}, {"type": "precision", "value": 0.6304330508019023, "name": "Precision Macro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2I3Y2RkNjU0NzlkYmJiNWYyZjZhNzIzZGE5ODU4NzYxYmQ0NTYxYzZkM2JiNTQwZTdkMmYxOTRmMDlmOGFkMiIsInZlcnNpb24iOjF9.iEt7Mq6a3TubFQfdC3OAxAiZDXp0bPGhN9JPzSfKl89_dxnKzDp0IrVzkt1HNLHR_S22Q75Tevqh3_G8Pp05Dg"}, {"type": "precision", "value": 0.6072386058981233, "name": "Precision Micro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmI4ZmEwOTY0NTViNTM1ZjcwY2E2ODRmNGJiMTg2ZDJmZTgyNGUxM2UwNjZjYzVmYTcxZGY4OGY3OTI4MzcyMSIsInZlcnNpb24iOjF9.Jn1OPD1ZxkblCqKT1CfeUYOt5Xb6CL6C2ZENLmvfYNzh-p0oHcIBgapfbCHc89oMSR-FhjQk_ME8f8A3eyy6CA"}, {"type": "precision", "value": 0.6320495884503851, "name": "Precision Weighted", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWU1NmI5MTk3ODY4Y2M5ZjYyNzMwZjRlZTAzNjFmNmUxNzgxYjVhODNjZDAwOTQyNDBlZDJkNDYyNzRlYzBmOSIsInZlcnNpb24iOjF9.ItCi8SouqOtM3P7c0KN5ifRpGOr1090aqo4zX4aVSlVOTq0iQj9_c3z0B_UAzFcr0qW7ObnvuiD8D5d-9EzkBg"}, {"type": "recall", "value": 0.6127303344145852, "name": "Recall Macro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTZmMGEzMGZjYTI2MzA0OGQ1Nzk5ZDRmNDNhMTRhOGMyM2I4Zjc2NzMzNmM2NjQ0YzVjZDY1Mzk1ZWI5Zjg2NiIsInZlcnNpb24iOjF9.fWCCNatB50nCptCbXopRjwxbWic6BvWIG6frUo_iXJVFXsi3Q_ik91_70fLgZc9NfhIpewpNoe4ETn0Gmps4Cw"}, {"type": "recall", "value": 0.6072386058981233, "name": "Recall Micro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGMwOWVmOWU4YzM2ZjMyY2NjNjljMDdjZTZmMjU5MjRiNDU0NmVhYWEzZmQ5MzUzMmRhMjdmZjhkNDU4ZTM2ZCIsInZlcnNpb24iOjF9.Sy2c29OhxT-x4UBSr9G7rfwtyqzYOX4KNRe2blonfOdKrqSfSEORY01Y67WweDiKdRvbECzI-DemJUXVtx-QCg"}, {"type": "recall", "value": 0.6072386058981233, "name": "Recall Weighted", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODgwODAwMDNkZWJlMzYyYmQwYzNmMGZiMzhjZjVjNGYyMTg3OWVjNjZmMzFhMDczNGEyMDAyODkwZTZhZWM4NSIsInZlcnNpb24iOjF9.8IxdbjDQHzcNW71RAMtKHzlviweLTQvYVQ4JlrqoZsV-8gyzxpyYOmDjUm3n6uQNfRLRpyvsT-E8ysLHPyMqDg"}, {"type": "f1", "value": 0.6010566054103847, "name": "F1 Macro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmMwMjg2YjZmYzMzYzlmNTk3OGIyMDc5NGUxNTFlNmNkNmU3MzU2ZWMyZjY0OGFhYmY5YTUyNDkxNjJiODIwNSIsInZlcnNpb24iOjF9.r1ylajrOC-Qu4QNdNnXzisjGlczTF_9tYpNEr8LYdTtdmJtRjNtNmElINneuaWX7XGN9TExdzmg7OWTwutjsAg"}, {"type": "f1", "value": 0.6072386058981233, "name": "F1 Micro", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDdhMGU1NDNjMzgxN2NjNWYzNTk5OWY5OTZhZWRjOGZkNjI4ZDA1YjI5ZWYxOWNmNDc4NmVhNjllMjUyMTFkMSIsInZlcnNpb24iOjF9.5G1km-a2_ssO_b3WTD8Ools29e6h8X8rjpClFN5Q_I4ADbPxKI2QbCfd5vl89CMHclignQ1_H6vqYbdTL9usDQ"}, {"type": "f1", "value": 0.5995456855334425, "name": "F1 Weighted", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjYyNTZjOTUzNmMxOTY3MzQzNjMxZGNhYTY3NTQ4Mjg3NWRlMjc2NmY1NjMxOWY0NTFiODlhZjA3ZTEzNGQ3MSIsInZlcnNpb24iOjF9.3iTI9IieFa3WJFr7ovDvO24IPScGB7WQk3Pw_Qxh32zKx5QyOwmZf_p2bgbEG6hBeCkR0KaMDvIiZXnbW6DBDQ"}, {"type": "loss", "value": 1.157181739807129, "name": "loss", "verified": true, "verifyToken": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmIxNTQyYWRkMjgxMTZkN2JhZTg5NDFiMDRlZGEzOGE5ZDIwYTE5NTU4YmU0NDUxOTE1MDQwMzFlMjQ5MDQ2YSIsInZlcnNpb24iOjF9.M-U6Dp-I-DEXZ3qSGLxYrCdQjgXi6DotHDgz1acjWnHIZWKPApy-n2194FZik1Tpv2AcJVe45tDRLxNSW3zVBg"}]}]}]}
transformers pytorch tf safetensors roberta text-classification indonesian-roberta-base-indonli id dataset:indonli arxiv:1907.11692 arxiv:2110.14566 license:mit model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #safetensors #roberta #text-classification #indonesian-roberta-base-indonli #id #dataset-indonli #arxiv-1907.11692 #arxiv-2110.14566 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 

Indonesian RoBERTa Base IndoNLI
-------------------------------


Indonesian RoBERTa Base IndoNLI is a natural language inference (NLI) model based on the RoBERTa model. The model was originally the pre-trained Indonesian RoBERTa Base model, which is then fine-tuned on 'IndoNLI''s dataset consisting of Indonesian Wikipedia, news, and Web articles [1].


After training, the model achieved an evaluation/dev accuracy of 77.06%. On the benchmark 'test\_lay' subset, the model achieved an accuracy of 74.24% and on the benchmark 'test\_expert' subset, the model achieved an accuracy of 61.66%.


Hugging Face's 'Trainer' class from the Transformers library was used to train the model. PyTorch was used as the backend framework during training, but the model remains compatible with other frameworks nonetheless.


Model
-----



Evaluation Results
------------------


The model was trained for 5 epochs, with a batch size of 16, a learning rate of 2e-5, a weight decay of 0.1, and a warmup ratio of 0.2, with linear annealing to 0. The best model was loaded at the end.



How to Use
----------


### As NLI Classifier


Disclaimer
----------


Do consider the biases which come from both the pre-trained RoBERTa model and the 'IndoNLI' dataset that may be carried over into the results of this model.


References
----------


[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). IndoNLI: A Natural Language Inference Dataset for Indonesian. *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*. Association for Computational Linguistics.


Author
------


Indonesian RoBERTa Base IndoNLI was trained and evaluated by Wilson Wongso. All computation and development are done on Google Colaboratory using their free GPU access.



### As NLI Classifier


Disclaimer
----------


Do consider the biases which come from both the pre-trained RoBERTa model and the 'IndoNLI' dataset that may be carried over into the results of this model.


References
----------


[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). IndoNLI: A Natural Language Inference Dataset for Indonesian. *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*. Association for Computational Linguistics.


Author
------


Indonesian RoBERTa Base IndoNLI was trained and evaluated by Wilson Wongso. All computation and development are done on Google Colaboratory using their free GPU access.
TAGS
#transformers #pytorch #tf #safetensors #roberta #text-classification #indonesian-roberta-base-indonli #id #dataset-indonli #arxiv-1907.11692 #arxiv-2110.14566 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
 ### As NLI Classifier


Disclaimer
----------


Do consider the biases which come from both the pre-trained RoBERTa model and the 'IndoNLI' dataset that may be carried over into the results of this model.


References
----------


[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). IndoNLI: A Natural Language Inference Dataset for Indonesian. *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*. Association for Computational Linguistics.


Author
------


Indonesian RoBERTa Base IndoNLI was trained and evaluated by Wilson Wongso. All computation and development are done on Google Colaboratory using their free GPU access.
passage: TAGS
#transformers #pytorch #tf #safetensors #roberta #text-classification #indonesian-roberta-base-indonli #id #dataset-indonli #arxiv-1907.11692 #arxiv-2110.14566 #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
### As NLI Classifier


Disclaimer
----------


Do consider the biases which come from both the pre-trained RoBERTa model and the 'IndoNLI' dataset that may be carried over into the results of this model.


References
----------


[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). IndoNLI: A Natural Language Inference Dataset for Indonesian. *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*. Association for Computational Linguistics.


Author
------


Indonesian RoBERTa Base IndoNLI was trained and evaluated by Wilson Wongso. All computation and development are done on Google Colaboratory using their free GPU access.