## Roberta-Base

This repo trains [roberta-base](https://huggingface.co/roberta-base) from scratch on the [Norwegian training subset of Oscar](https://oscar-corpus.com/) containing roughly 4.7 GB of data according to [this](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling) example.

Training is done on a TPUv3-8 in Flax. More statistics on the training run can be found under [tf.hub](https://tensorboard.dev/experiment/GdYmdak2TWeVz0DDRYOrrg).

{}
transformers pytorch jax roberta fill-mask autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #jax #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 

## Roberta-Base

This repo trains roberta-base from scratch on the Norwegian training subset of Oscar containing roughly 4.7 GB of data according to this example.

Training is done on a TPUv3-8 in Flax. More statistics on the training run can be found under URL.

## Roberta-Base

This repo trains roberta-base from scratch on the Norwegian training subset of Oscar containing roughly 4.7 GB of data according to this example.

Training is done on a TPUv3-8 in Flax. More statistics on the training run can be found under URL.
TAGS
#transformers #pytorch #jax #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 
 ## Roberta-Base

This repo trains roberta-base from scratch on the Norwegian training subset of Oscar containing roughly 4.7 GB of data according to this example.

Training is done on a TPUv3-8 in Flax. More statistics on the training run can be found under URL.
passage: TAGS
#transformers #pytorch #jax #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 
## Roberta-Base

This repo trains roberta-base from scratch on the Norwegian training subset of Oscar containing roughly 4.7 GB of data according to this example.

Training is done on a TPUv3-8 in Flax. More statistics on the training run can be found under URL.