
# mBERTu

A Maltese multilingual model pre-trained on the Korpus Malti v4.0 using multilingual BERT as the initial checkpoint.


## License

This work is licensed under a
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
Permissions beyond the scope of this license may be available at [https://mlrs.research.um.edu.mt/](https://mlrs.research.um.edu.mt/).

[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]

[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/
[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png

## Citation

This work was first presented in [Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese](https://aclanthology.org/2022.deeplo-1.10/).
Cite it as follows: 

```bibtex
@inproceedings{BERTu,
    title = "Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and {BERT} Models for {M}altese",
    author = "Micallef, Kurt  and
              Gatt, Albert  and
              Tanti, Marc  and
              van der Plas, Lonneke  and
              Borg, Claudia",
    booktitle = "Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing",
    month = jul,
    year = "2022",
    address = "Hybrid",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.deeplo-1.10",
    doi = "10.18653/v1/2022.deeplo-1.10",
    pages = "90--101",
}
```

{"language": ["mt"], "license": "cc-by-nc-sa-4.0", "datasets": ["MLRS/korpus_malti"], "widget": [{"text": "Malta huwa pajji\u017c fl-[MASK]."}], "model-index": [{"name": "mBERTu", "results": [{"task": {"type": "dependency-parsing", "name": "Dependency Parsing"}, "dataset": {"name": "Maltese Universal Dependencies Treebank (MUDT)", "type": "universal_dependencies", "args": "mt_mudt"}, "metrics": [{"type": "uas", "value": 92.1, "name": "Unlabelled Attachment Score"}, {"type": "las", "value": 87.87, "name": "Labelled Attachment Score"}]}, {"task": {"type": "part-of-speech-tagging", "name": "Part-of-Speech Tagging"}, "dataset": {"name": "MLRS POS dataset", "type": "mlrs_pos"}, "metrics": [{"type": "accuracy", "value": 98.66, "name": "UPOS Accuracy", "args": "upos"}, {"type": "accuracy", "value": 98.58, "name": "XPOS Accuracy", "args": "xpos"}]}, {"task": {"type": "named-entity-recognition", "name": "Named Entity Recognition"}, "dataset": {"name": "WikiAnn (Maltese)", "type": "wikiann", "args": "mt"}, "metrics": [{"type": "f1", "value": 86.6, "name": "Span-based F1", "args": "span"}]}, {"task": {"type": "sentiment-analysis", "name": "Sentiment Analysis"}, "dataset": {"name": "Maltese Sentiment Analysis Dataset", "type": "mt-sentiment-analysis"}, "metrics": [{"type": "f1", "value": 76.79, "name": "Macro-averaged F1", "args": "macro"}]}]}]}
transformers pytorch bert fill-mask mt dataset:MLRS/korpus_malti license:cc-by-nc-sa-4.0 model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #mt #dataset-MLRS/korpus_malti #license-cc-by-nc-sa-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 


# mBERTu

A Maltese multilingual model pre-trained on the Korpus Malti v4.0 using multilingual BERT as the initial checkpoint.


## License

This work is licensed under a
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
Permissions beyond the scope of this license may be available at URL

[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]

[cc-by-nc-sa]: URL
[cc-by-nc-sa-image]: URL

This work was first presented in Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese.
Cite it as follows: 



# mBERTu

A Maltese multilingual model pre-trained on the Korpus Malti v4.0 using multilingual BERT as the initial checkpoint. ## License

This work is licensed under a
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
Permissions beyond the scope of this license may be available at URL

[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]

[cc-by-nc-sa]: URL
[cc-by-nc-sa-image]: URL

This work was first presented in Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese.
Cite it as follows:
TAGS
#transformers #pytorch #bert #fill-mask #mt #dataset-MLRS/korpus_malti #license-cc-by-nc-sa-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 
 # mBERTu

A Maltese multilingual model pre-trained on the Korpus Malti v4.0 using multilingual BERT as the initial checkpoint. ## License

This work is licensed under a
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
Permissions beyond the scope of this license may be available at URL

[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]

[cc-by-nc-sa]: URL
[cc-by-nc-sa-image]: URL

This work was first presented in Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese.
Cite it as follows:
passage: TAGS
#transformers #pytorch #bert #fill-mask #mt #dataset-MLRS/korpus_malti #license-cc-by-nc-sa-4.0 #model-index #autotrain_compatible #endpoints_compatible #region-us 
# mBERTu

A Maltese multilingual model pre-trained on the Korpus Malti v4.0 using multilingual BERT as the initial checkpoint.## License

This work is licensed under a
[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].
Permissions beyond the scope of this license may be available at URL

[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]

[cc-by-nc-sa]: URL
[cc-by-nc-sa-image]: URL

This work was first presented in Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese.
Cite it as follows: