metadata
library_name: transformers
license: apache-2.0
base_model: google/t5-efficient-tiny-nh8
tags:
- generated_from_trainer
model-index:
- name: t5-efficient-tiny-nh8-summarizer
results: []
t5-efficient-tiny-nh8-summarizer
This model is a fine-tuned version of google/t5-efficient-tiny-nh8 on shorecode/summary-collection-60k-rows. It achieves the following results on the evaluation set:
- Loss: 0.7583
Model description
A general purpose text summarizer
Intended uses & limitations
A general purpose text summarizer
Training and evaluation data
Trained and evaluated on shorecode/summary-collection-60k-rows
Training procedure
Trained using the Gradio SDK on Hugging Face Spaces using shared Zero GPU(s)
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 7.000000000000001e-05
- train_batch_size: 70
- eval_batch_size: 70
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.1522 | 0.2328 | 200 | 0.9863 |
| 0.9677 | 0.4657 | 400 | 0.9158 |
| 0.9143 | 0.6985 | 600 | 0.8762 |
| 0.8894 | 0.9313 | 800 | 0.8478 |
| 0.8586 | 1.1641 | 1000 | 0.8262 |
| 0.8382 | 1.3970 | 1200 | 0.8079 |
| 0.8198 | 1.6298 | 1400 | 0.7938 |
| 0.805 | 1.8626 | 1600 | 0.7823 |
| 0.8035 | 2.0955 | 1800 | 0.7727 |
| 0.7897 | 2.3283 | 2000 | 0.7661 |
| 0.7849 | 2.5611 | 2200 | 0.7607 |
| 0.7781 | 2.7939 | 2400 | 0.7583 |
Framework versions
- Transformers 4.47.0
- Pytorch 2.4.0+cu121
- Datasets 3.0.0
- Tokenizers 0.21.0