pytorch tensorboard t5 text2text-generation dataset:xsum transformers generated_from_trainer model-index autotrain_compatible
---
tags:
- generated_from_trainer
datasets:
- xsum
metrics:
- rouge
model-index:
- name: t5-small-finetuned_xsum
  results:
  - task:
      name: Sequence-to-sequence Language Modeling
      type: text2text-generation
    dataset:
      name: xsum
      type: xsum
      args: default
    metrics:
    - name: Rouge1
      type: rouge
      value: 34.0559
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-finetuned_xsum

This model is a fine-tuned version of [pki/t5-small-finetuned_xsum](https://huggingface.co/pki/t5-small-finetuned_xsum) on the xsum dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0479
- Rouge1: 34.0559
- Rouge2: 12.7506
- Rougel: 27.6762
- Rougelsum: 27.68
- Gen Len: 18.7924

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 2.1176        | 1.0   | 12753  | 2.0913          | 33.1548 | 11.8434 | 26.7805 | 26.7751   | 18.7805 |
| 2.1019        | 2.0   | 25506  | 2.0875          | 33.231  | 11.9329 | 26.8674 | 26.861    | 18.7992 |
| 2.1044        | 3.0   | 38259  | 2.0846          | 33.3643 | 11.9807 | 26.9817 | 26.9764   | 18.773  |
| 2.0874        | 4.0   | 51012  | 2.0832          | 33.3562 | 12.0681 | 27.0178 | 27.0189   | 18.7988 |
| 2.0791        | 5.0   | 63765  | 2.0803          | 33.38   | 12.081  | 27.0368 | 27.0344   | 18.7844 |
| 2.0894        | 6.0   | 76518  | 2.0787          | 33.2549 | 11.9662 | 26.8674 | 26.8669   | 18.7975 |
| 2.0802        | 7.0   | 89271  | 2.0777          | 33.3978 | 12.0828 | 27.0461 | 27.0443   | 18.7757 |
| 2.0719        | 8.0   | 102024 | 2.0743          | 33.4083 | 12.1141 | 27.0523 | 27.0457   | 18.7928 |
| 2.0782        | 9.0   | 114777 | 2.0748          | 33.3673 | 12.1637 | 27.0696 | 27.0663   | 18.7902 |
| 2.0736        | 10.0  | 127530 | 2.0713          | 33.5771 | 12.2219 | 27.1707 | 27.1706   | 18.7945 |
| 2.0816        | 11.0  | 140283 | 2.0703          | 33.5099 | 12.2069 | 27.1822 | 27.1835   | 18.8002 |
| 2.057         | 12.0  | 153036 | 2.0693          | 33.5853 | 12.2427 | 27.2096 | 27.2109   | 18.806  |
| 2.0584        | 13.0  | 165789 | 2.0676          | 33.4883 | 12.2674 | 27.1582 | 27.154    | 18.7857 |
| 2.0475        | 14.0  | 178542 | 2.0662          | 33.5529 | 12.2765 | 27.1897 | 27.1901   | 18.79   |
| 2.0426        | 15.0  | 191295 | 2.0643          | 33.6543 | 12.3545 | 27.2946 | 27.2928   | 18.8036 |
| 2.0373        | 16.0  | 204048 | 2.0648          | 33.6671 | 12.349  | 27.2649 | 27.2707   | 18.7905 |
| 2.0178        | 17.0  | 216801 | 2.0637          | 33.6794 | 12.4545 | 27.3015 | 27.3079   | 18.7948 |
| 2.0235        | 18.0  | 229554 | 2.0626          | 33.7635 | 12.423  | 27.3475 | 27.3446   | 18.7892 |
| 2.0296        | 19.0  | 242307 | 2.0622          | 33.7574 | 12.4651 | 27.3879 | 27.3882   | 18.8134 |
| 2.0319        | 20.0  | 255060 | 2.0595          | 33.9093 | 12.5389 | 27.5003 | 27.5001   | 18.7915 |
| 2.0208        | 21.0  | 267813 | 2.0583          | 33.7875 | 12.4912 | 27.4243 | 27.4332   | 18.7982 |
| 2.0151        | 22.0  | 280566 | 2.0581          | 33.8516 | 12.4805 | 27.46   | 27.4647   | 18.816  |
| 2.0188        | 23.0  | 293319 | 2.0575          | 33.7744 | 12.4548 | 27.381  | 27.382    | 18.802  |
| 2.0087        | 24.0  | 306072 | 2.0579          | 33.8953 | 12.4984 | 27.4675 | 27.4727   | 18.7819 |
| 2.0186        | 25.0  | 318825 | 2.0557          | 33.7766 | 12.4414 | 27.4025 | 27.4024   | 18.8005 |
| 2.0051        | 26.0  | 331578 | 2.0555          | 33.8973 | 12.5796 | 27.5338 | 27.5339   | 18.8153 |
| 2.0024        | 27.0  | 344331 | 2.0557          | 33.8709 | 12.5116 | 27.4684 | 27.4664   | 18.7911 |
| 1.9947        | 28.0  | 357084 | 2.0545          | 33.8499 | 12.5242 | 27.4677 | 27.4716   | 18.8025 |
| 1.9931        | 29.0  | 369837 | 2.0545          | 33.7957 | 12.5272 | 27.4129 | 27.4174   | 18.8    |
| 1.9826        | 30.0  | 382590 | 2.0548          | 33.9723 | 12.6665 | 27.5598 | 27.5662   | 18.7958 |
| 1.999         | 31.0  | 395343 | 2.0522          | 33.9702 | 12.6435 | 27.5788 | 27.579    | 18.795  |
| 1.9872        | 32.0  | 408096 | 2.0525          | 33.9546 | 12.638  | 27.5985 | 27.5949   | 18.7976 |
| 1.991         | 33.0  | 420849 | 2.0520          | 33.9792 | 12.6073 | 27.5686 | 27.5707   | 18.8056 |
| 2.0044        | 34.0  | 433602 | 2.0504          | 34.0736 | 12.6511 | 27.647  | 27.6472   | 18.8093 |
| 1.9972        | 35.0  | 446355 | 2.0513          | 34.0506 | 12.711  | 27.6533 | 27.6537   | 18.7984 |
| 1.9901        | 36.0  | 459108 | 2.0504          | 33.9991 | 12.638  | 27.626  | 27.6272   | 18.7996 |
| 1.9742        | 37.0  | 471861 | 2.0507          | 33.9357 | 12.6636 | 27.5673 | 27.5716   | 18.8064 |
| 1.984         | 38.0  | 484614 | 2.0502          | 33.9476 | 12.6589 | 27.58   | 27.5813   | 18.8037 |
| 1.9864        | 39.0  | 497367 | 2.0499          | 34.0733 | 12.7198 | 27.6926 | 27.6992   | 18.8061 |
| 1.9734        | 40.0  | 510120 | 2.0492          | 33.9483 | 12.6486 | 27.5571 | 27.5598   | 18.8033 |
| 1.9895        | 41.0  | 522873 | 2.0490          | 33.9753 | 12.684  | 27.6058 | 27.6086   | 18.8011 |
| 1.964         | 42.0  | 535626 | 2.0487          | 33.9528 | 12.6376 | 27.576  | 27.5824   | 18.7919 |
| 1.9849        | 43.0  | 548379 | 2.0487          | 33.9868 | 12.6936 | 27.6116 | 27.6158   | 18.7966 |
| 1.9798        | 44.0  | 561132 | 2.0491          | 34.0379 | 12.7161 | 27.6227 | 27.6315   | 18.7889 |
| 1.9837        | 45.0  | 573885 | 2.0473          | 34.0046 | 12.6559 | 27.5931 | 27.5988   | 18.7996 |
| 1.9556        | 46.0  | 586638 | 2.0483          | 34.0378 | 12.712  | 27.6346 | 27.6446   | 18.7942 |
| 1.9844        | 47.0  | 599391 | 2.0479          | 34.0301 | 12.7121 | 27.6492 | 27.6554   | 18.7999 |
| 1.9869        | 48.0  | 612144 | 2.0474          | 34.0463 | 12.7151 | 27.6542 | 27.6604   | 18.7919 |
| 1.9851        | 49.0  | 624897 | 2.0476          | 34.0549 | 12.7384 | 27.6542 | 27.6555   | 18.7924 |
| 1.9912        | 50.0  | 637650 | 2.0479          | 34.0559 | 12.7506 | 27.6762 | 27.68     | 18.7924 |



### Framework versions

- Transformers 4.12.0.dev0
- Pytorch 1.10.1
- Datasets 1.14.0
- Tokenizers 0.10.3
