
## BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters `L=8` and `H=512`. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765

{"language": ["lu"], "license": "mit", "tags": ["text", "MLM"]}
transformers tf bert fill-mask text MLM lu license:mit autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #tf #bert #fill-mask #text #MLM #lu #license-mit #autotrain_compatible #endpoints_compatible #region-us 


## BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters 'L=8' and 'H=512'. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765

## BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters 'L=8' and 'H=512'. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765
TAGS
#transformers #tf #bert #fill-mask #text #MLM #lu #license-mit #autotrain_compatible #endpoints_compatible #region-us 
 ## BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters 'L=8' and 'H=512'. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765
passage: TAGS
#transformers #tf #bert #fill-mask #text #MLM #lu #license-mit #autotrain_compatible #endpoints_compatible #region-us 
## BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters 'L=8' and 'H=512'. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

- Final train loss: 4.230
- Final train perplexity: 68.726
- Final validation loss: 4.074
- Final validation perplexity: 58.765