
# ALBERT Base Spanish

This is an [ALBERT](https://github.com/google-research/albert) model trained on a [big spanish corpora](https://github.com/josecannete/spanish-corpora).
The model was trained on a single TPU v3-8 with the following hyperparameters and steps/time:
- LR: 0.0008838834765
- Batch Size: 960
- Warmup ratio: 0.00625
- Warmup steps: 53333.33333
- Goal steps: 8533333.333
- Total steps: 3650000
- Total training time (aprox): 70.4 days.

## Training loss

![https://drive.google.com/uc?export=view&id=1IsxcgMwd7Hl-3bSnNl8W9jUrHJeHtZql](https://drive.google.com/uc?export=view&id=1IsxcgMwd7Hl-3bSnNl8W9jUrHJeHtZql)
{"language": ["es"], "tags": ["albert", "spanish", "OpenCENIA"], "datasets": ["large_spanish_corpus"]}
transformers pytorch tf albert pretraining spanish OpenCENIA es dataset:large_spanish_corpus endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #albert #pretraining #spanish #OpenCENIA #es #dataset-large_spanish_corpus #endpoints_compatible #region-us 


# ALBERT Base Spanish

This is an ALBERT model trained on a big spanish corpora.
The model was trained on a single TPU v3-8 with the following hyperparameters and steps/time:
- LR: 0.0008838834765
- Batch Size: 960
- Warmup ratio: 0.00625
- Warmup steps: 53333.33333
- Goal steps: 8533333.333
- Total steps: 3650000
- Total training time (aprox): 70.4 days.

## Training loss

!URL
# ALBERT Base Spanish

This is an ALBERT model trained on a big spanish corpora.
The model was trained on a single TPU v3-8 with the following hyperparameters and steps/time:
- LR: 0.0008838834765
- Batch Size: 960
- Warmup ratio: 0.00625
- Warmup steps: 53333.33333
- Goal steps: 8533333.333
- Total steps: 3650000
- Total training time (aprox): 70.4 days. ## Training loss

!URL
TAGS
#transformers #pytorch #tf #albert #pretraining #spanish #OpenCENIA #es #dataset-large_spanish_corpus #endpoints_compatible #region-us 
 # ALBERT Base Spanish

This is an ALBERT model trained on a big spanish corpora.
The model was trained on a single TPU v3-8 with the following hyperparameters and steps/time:
- LR: 0.0008838834765
- Batch Size: 960
- Warmup ratio: 0.00625
- Warmup steps: 53333.33333
- Goal steps: 8533333.333
- Total steps: 3650000
- Total training time (aprox): 70.4 days. ## Training loss

!URL
passage: TAGS
#transformers #pytorch #tf #albert #pretraining #spanish #OpenCENIA #es #dataset-large_spanish_corpus #endpoints_compatible #region-us 
# ALBERT Base Spanish

This is an ALBERT model trained on a big spanish corpora.
The model was trained on a single TPU v3-8 with the following hyperparameters and steps/time:
- LR: 0.0008838834765
- Batch Size: 960
- Warmup ratio: 0.00625
- Warmup steps: 53333.33333
- Goal steps: 8533333.333
- Total steps: 3650000
- Total training time (aprox): 70.4 days.## Training loss

!URL