
# byt5-basque

Pretrained from scratch on Euskara (Basque language) 
with ByT5, Google's new byte-level tokenizer strategy.

Corpus: eu.wikipedia.org as of March 2020 (TFDS)

Pretraining Notebook: https://colab.research.google.com/drive/19Afq7CI6cOi1DaTpnQhBbEbnBzLSFHbH

## Todos

Fine-tuning

The Wikipedia corpus is small for this language compared to web crawls. In the future I would add
OSCAR, if I can rewrite the script to accept those
as one TFDS dataset.


{"language": "eu"}
transformers pytorch tf t5 text2text-generation eu autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tf #t5 #text2text-generation #eu #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# byt5-basque

Pretrained from scratch on Euskara (Basque language) 
with ByT5, Google's new byte-level tokenizer strategy.

Corpus: URL as of March 2020 (TFDS)

Pretraining Notebook: URL

## Todos

Fine-tuning

The Wikipedia corpus is small for this language compared to web crawls. In the future I would add
OSCAR, if I can rewrite the script to accept those
as one TFDS dataset.


# byt5-basque

Pretrained from scratch on Euskara (Basque language) 
with ByT5, Google's new byte-level tokenizer strategy.

Corpus: URL as of March 2020 (TFDS)

Pretraining Notebook: URL ## Todos

Fine-tuning

The Wikipedia corpus is small for this language compared to web crawls. In the future I would add
OSCAR, if I can rewrite the script to accept those
as one TFDS dataset.
TAGS
#transformers #pytorch #tf #t5 #text2text-generation #eu #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # byt5-basque

Pretrained from scratch on Euskara (Basque language) 
with ByT5, Google's new byte-level tokenizer strategy.

Corpus: URL as of March 2020 (TFDS)

Pretraining Notebook: URL ## Todos

Fine-tuning

The Wikipedia corpus is small for this language compared to web crawls. In the future I would add
OSCAR, if I can rewrite the script to accept those
as one TFDS dataset.
passage: TAGS
#transformers #pytorch #tf #t5 #text2text-generation #eu #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# byt5-basque

Pretrained from scratch on Euskara (Basque language) 
with ByT5, Google's new byte-level tokenizer strategy.

Corpus: URL as of March 2020 (TFDS)

Pretraining Notebook: URL## Todos

Fine-tuning

The Wikipedia corpus is small for this language compared to web crawls. In the future I would add
OSCAR, if I can rewrite the script to accept those
as one TFDS dataset.