pytorch t5 text2text-generation transformers generated_from_trainer license:apache-2.0 model-index autotrain_compatible
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: tunixx.20
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# tunixx.20

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6492
- Bleu: 62.3581

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 420
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Bleu    |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| No log        | 0.6   | 100  | 2.1821          | 14.2209 |
| No log        | 1.19  | 200  | 1.9019          | 17.6606 |
| No log        | 1.79  | 300  | 1.6948          | 19.7423 |
| No log        | 2.38  | 400  | 1.5505          | 23.6162 |
| 1.9238        | 2.98  | 500  | 1.4374          | 27.3088 |
| 1.9238        | 3.57  | 600  | 1.3460          | 31.0185 |
| 1.9238        | 4.17  | 700  | 1.2517          | 33.3477 |
| 1.9238        | 4.76  | 800  | 1.1763          | 33.9847 |
| 1.9238        | 5.36  | 900  | 1.1152          | 34.1613 |
| 1.3121        | 5.95  | 1000 | 1.0539          | 35.4759 |
| 1.3121        | 6.55  | 1100 | 1.0081          | 36.6102 |
| 1.3121        | 7.14  | 1200 | 0.9568          | 37.5106 |
| 1.3121        | 7.74  | 1300 | 0.9156          | 38.0362 |
| 1.3121        | 8.33  | 1400 | 0.8857          | 38.4678 |
| 1.0132        | 8.93  | 1500 | 0.8527          | 38.8540 |
| 1.0132        | 9.52  | 1600 | 0.8216          | 39.4236 |
| 1.0132        | 10.12 | 1700 | 0.7954          | 39.3181 |
| 1.0132        | 10.71 | 1800 | 0.7741          | 39.7601 |
| 1.0132        | 11.31 | 1900 | 0.7551          | 40.0916 |
| 0.8567        | 11.9  | 2000 | 0.7386          | 41.1072 |
| 0.8567        | 12.5  | 2100 | 0.7231          | 41.3821 |
| 0.8567        | 13.1  | 2200 | 0.7103          | 41.8838 |
| 0.8567        | 13.69 | 2300 | 0.6982          | 42.0218 |
| 0.8567        | 14.29 | 2400 | 0.6870          | 41.7599 |
| 0.7764        | 14.88 | 2500 | 0.6786          | 42.3989 |
| 0.7764        | 15.48 | 2600 | 0.6709          | 42.7624 |
| 0.7764        | 16.07 | 2700 | 0.6634          | 42.9174 |
| 0.7764        | 16.67 | 2800 | 0.6567          | 42.9174 |
| 0.7764        | 17.26 | 2900 | 0.6525          | 43.4440 |
| 0.7282        | 17.86 | 3000 | 0.6492          | 43.4440 |
| 0.7282        | 18.45 | 3100 | 0.6468          | 43.6901 |
| 0.7282        | 19.05 | 3200 | 0.6445          | 43.5582 |
| 0.7282        | 19.64 | 3300 | 0.6435          | 43.5582 |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0+cu102
- Datasets 2.1.0
- Tokenizers 0.12.1
