pytorch bart text2text-generation dataset:yawnick/QMSum transformers generated_from_trainer license:apache-2.0 model-index autotrain_compatible
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: bart-qmsum-meeting-summarization
  results: []
datasets:
- yawnick/QMSum
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bart-qmsum-meeting-summarization

This model is a fine-tuned version of [sshleifer/distilbart-cnn-12-6](https://huggingface.co/sshleifer/distilbart-cnn-12-6) on the QMSum dataset.
It achieves the following results on the evaluation set:
- Loss: 4.3354
- Rouge1: 39.5539
- Rouge2: 12.1134
- Rougel: 23.9163
- Rougelsum: 36.0299
- Gen Len: 117.225

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-07
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 200
- label_smoothing_factor: 0.1

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 5.5573        | 2.17   | 100  | 5.4074          | 23.6282 | 4.1122  | 14.584  | 21.2263   | 84.75   |
| 5.4721        | 4.35   | 200  | 5.2899          | 24.61   | 4.272   | 15.2096 | 22.2997   | 87.2    |
| 5.3407        | 6.52   | 300  | 5.1360          | 25.8272 | 4.3314  | 15.9926 | 23.3416   | 87.95   |
| 5.1527        | 8.7    | 400  | 4.9751          | 27.7207 | 5.31    | 16.7055 | 24.8357   | 88.35   |
| 5.0058        | 10.87  | 500  | 4.8372          | 30.1847 | 6.8615  | 18.934  | 27.2424   | 89.95   |
| 4.8807        | 13.04  | 600  | 4.7488          | 33.1208 | 9.1784  | 20.655  | 30.1198   | 101.3   |
| 4.7931        | 15.22  | 700  | 4.6891          | 33.2266 | 8.4253  | 20.0334 | 30.4093   | 108.925 |
| 4.7272        | 17.39  | 800  | 4.6467          | 35.0475 | 9.326   | 21.0655 | 31.8413   | 111.7   |
| 4.6904        | 19.57  | 900  | 4.6102          | 34.869  | 9.6046  | 21.395  | 32.4346   | 115.05  |
| 4.6547        | 21.74  | 1000 | 4.5829          | 36.3392 | 10.9936 | 22.1524 | 33.6863   | 119.875 |
| 4.594         | 23.91  | 1100 | 4.5602          | 35.9717 | 10.3827 | 21.6118 | 32.8302   | 119.5   |
| 4.5714        | 26.09  | 1200 | 4.5424          | 36.3656 | 10.6282 | 22.2187 | 33.6494   | 118.0   |
| 4.542         | 28.26  | 1300 | 4.5256          | 36.7386 | 10.615  | 22.2487 | 34.1927   | 115.675 |
| 4.5092        | 30.43  | 1400 | 4.5116          | 37.1597 | 10.7751 | 22.6747 | 34.396    | 118.55  |
| 4.5031        | 32.61  | 1500 | 4.4981          | 37.6108 | 10.9732 | 22.8342 | 34.6833   | 117.125 |
| 4.4682        | 34.78  | 1600 | 4.4875          | 37.5057 | 11.1328 | 22.8973 | 34.7114   | 117.65  |
| 4.4387        | 36.96  | 1700 | 4.4775          | 38.1278 | 11.3597 | 23.1307 | 35.1869   | 115.65  |
| 4.4085        | 39.13  | 1800 | 4.4682          | 37.9578 | 11.4355 | 23.1149 | 35.4961   | 119.6   |
| 4.4166        | 41.3   | 1900 | 4.4592          | 38.1467 | 11.3208 | 23.045  | 35.0824   | 120.05  |
| 4.3971        | 43.48  | 2000 | 4.4517          | 37.9922 | 11.5071 | 23.3983 | 34.6918   | 114.425 |
| 4.3638        | 45.65  | 2100 | 4.4438          | 38.1666 | 11.4985 | 23.5518 | 35.1484   | 117.2   |
| 4.3522        | 47.83  | 2200 | 4.4377          | 37.7572 | 11.3984 | 23.4437 | 35.0453   | 113.725 |
| 4.3398        | 50.0   | 2300 | 4.4320          | 38.5833 | 11.4575 | 23.6411 | 35.3437   | 116.125 |
| 4.3341        | 52.17  | 2400 | 4.4247          | 38.2705 | 12.0374 | 23.5807 | 34.9985   | 110.8   |
| 4.3024        | 54.35  | 2500 | 4.4201          | 39.0206 | 12.2041 | 23.4394 | 35.6291   | 114.5   |
| 4.3117        | 56.52  | 2600 | 4.4147          | 38.6555 | 12.1079 | 23.5655 | 35.5287   | 111.325 |
| 4.2659        | 58.7   | 2700 | 4.4107          | 39.2235 | 12.025  | 23.934  | 36.2243   | 113.3   |
| 4.2946        | 60.87  | 2800 | 4.4055          | 39.0301 | 12.1833 | 23.8999 | 36.0487   | 110.325 |
| 4.2431        | 63.04  | 2900 | 4.4009          | 39.0498 | 12.3215 | 23.9686 | 36.0277   | 112.775 |
| 4.2439        | 65.22  | 3000 | 4.3968          | 38.8786 | 12.0985 | 23.8308 | 35.8575   | 115.175 |
| 4.2244        | 67.39  | 3100 | 4.3922          | 38.7614 | 12.1721 | 23.7736 | 35.6744   | 113.55  |
| 4.235         | 69.57  | 3200 | 4.3895          | 38.6858 | 11.3994 | 23.6392 | 35.3456   | 114.125 |
| 4.2064        | 71.74  | 3300 | 4.3859          | 39.0258 | 12.0435 | 24.2528 | 35.8378   | 113.5   |
| 4.1934        | 73.91  | 3400 | 4.3835          | 39.0467 | 11.5556 | 23.6704 | 35.5643   | 111.5   |
| 4.1859        | 76.09  | 3500 | 4.3800          | 38.776  | 11.729  | 24.1254 | 35.3894   | 112.9   |
| 4.1762        | 78.26  | 3600 | 4.3775          | 38.9465 | 11.9112 | 23.8123 | 35.5453   | 114.125 |
| 4.1848        | 80.43  | 3700 | 4.3744          | 39.2783 | 11.6539 | 23.8236 | 35.8465   | 110.225 |
| 4.1386        | 82.61  | 3800 | 4.3730          | 38.8894 | 11.4784 | 23.7534 | 35.5464   | 113.15  |
| 4.1483        | 84.78  | 3900 | 4.3710          | 39.2734 | 12.0285 | 23.8171 | 35.6884   | 115.95  |
| 4.1428        | 86.96  | 4000 | 4.3688          | 39.6134 | 12.0616 | 23.7454 | 36.0363   | 113.375 |
| 4.133         | 89.13  | 4100 | 4.3663          | 38.935  | 11.4781 | 23.8766 | 35.4061   | 114.15  |
| 4.1211        | 91.3   | 4200 | 4.3648          | 39.1488 | 11.8399 | 23.9935 | 35.3107   | 113.975 |
| 4.1076        | 93.48  | 4300 | 4.3650          | 38.9764 | 11.9963 | 23.4994 | 35.7214   | 116.25  |
| 4.121         | 95.65  | 4400 | 4.3597          | 38.9418 | 11.8416 | 24.0272 | 35.6597   | 111.325 |
| 4.0936        | 97.83  | 4500 | 4.3602          | 39.266  | 12.5616 | 24.2046 | 36.1883   | 114.275 |
| 4.0841        | 100.0  | 4600 | 4.3588          | 39.4659 | 12.2132 | 24.0521 | 36.249    | 115.475 |
| 4.0768        | 102.17 | 4700 | 4.3578          | 39.4167 | 12.0587 | 24.025  | 35.9668   | 114.375 |
| 4.0711        | 104.35 | 4800 | 4.3541          | 39.6943 | 12.1095 | 24.0925 | 36.3496   | 115.65  |
| 4.072         | 106.52 | 4900 | 4.3539          | 40.2024 | 12.4618 | 24.2863 | 36.8844   | 113.475 |
| 4.0646        | 108.7  | 5000 | 4.3540          | 39.4299 | 11.8085 | 23.686  | 36.0454   | 113.975 |
| 4.0508        | 110.87 | 5100 | 4.3517          | 39.9217 | 11.9379 | 24.2299 | 36.6362   | 115.5   |
| 4.0549        | 113.04 | 5200 | 4.3498          | 40.3496 | 12.2558 | 24.0271 | 36.9715   | 112.5   |
| 4.0428        | 115.22 | 5300 | 4.3497          | 40.1349 | 12.0628 | 24.0622 | 36.9169   | 113.95  |
| 4.0391        | 117.39 | 5400 | 4.3480          | 40.1209 | 12.3587 | 24.3456 | 36.8411   | 116.025 |
| 4.0195        | 119.57 | 5500 | 4.3474          | 39.5209 | 12.1325 | 24.2622 | 36.4357   | 111.975 |
| 4.0054        | 121.74 | 5600 | 4.3468          | 40.2885 | 12.4453 | 24.2373 | 36.932    | 117.375 |
| 4.0286        | 123.91 | 5700 | 4.3465          | 39.3943 | 11.8399 | 23.9786 | 35.991    | 116.475 |
| 4.005         | 126.09 | 5800 | 4.3459          | 38.7442 | 11.7408 | 23.8948 | 35.3673   | 117.625 |
| 3.991         | 128.26 | 5900 | 4.3444          | 39.6276 | 12.1549 | 23.9542 | 36.3832   | 115.675 |
| 4.0137        | 130.43 | 6000 | 4.3427          | 39.8331 | 12.2687 | 24.187  | 36.6144   | 115.475 |
| 3.9755        | 132.61 | 6100 | 4.3438          | 39.1907 | 12.1033 | 24.2339 | 35.9126   | 114.525 |
| 4.0134        | 134.78 | 6200 | 4.3422          | 39.4298 | 11.862  | 24.0847 | 35.5744   | 115.025 |
| 3.9935        | 136.96 | 6300 | 4.3416          | 39.4158 | 11.6968 | 23.9636 | 35.8155   | 114.35  |
| 3.9606        | 139.13 | 6400 | 4.3409          | 39.1239 | 11.7046 | 23.6846 | 36.0431   | 114.775 |
| 3.9834        | 141.3  | 6500 | 4.3404          | 39.6375 | 12.2746 | 24.2636 | 36.1425   | 116.175 |
| 3.9687        | 143.48 | 6600 | 4.3409          | 39.1494 | 12.1404 | 24.0778 | 35.4932   | 118.05  |
| 3.9861        | 145.65 | 6700 | 4.3394          | 39.6258 | 12.2497 | 23.9662 | 36.4054   | 116.8   |
| 3.9755        | 147.83 | 6800 | 4.3400          | 39.3121 | 11.7831 | 23.6584 | 35.9636   | 118.125 |
| 3.9591        | 150.0  | 6900 | 4.3390          | 39.6957 | 11.9406 | 24.0599 | 36.3021   | 114.9   |
| 3.9599        | 152.17 | 7000 | 4.3389          | 39.4271 | 11.4159 | 24.1437 | 35.9056   | 115.8   |
| 3.9456        | 154.35 | 7100 | 4.3384          | 39.4862 | 11.726  | 23.883  | 35.9839   | 116.375 |
| 3.9341        | 156.52 | 7200 | 4.3386          | 39.6915 | 11.8028 | 24.346  | 36.406    | 116.425 |
| 3.9648        | 158.7  | 7300 | 4.3383          | 39.9311 | 11.7135 | 23.985  | 36.2617   | 118.075 |
| 3.9486        | 160.87 | 7400 | 4.3372          | 39.8375 | 12.0014 | 24.0969 | 36.5902   | 118.8   |
| 3.9533        | 163.04 | 7500 | 4.3371          | 40.2678 | 12.3137 | 24.1916 | 37.1632   | 118.075 |
| 3.9344        | 165.22 | 7600 | 4.3369          | 39.5588 | 11.6805 | 24.1474 | 36.2021   | 114.875 |
| 3.9314        | 167.39 | 7700 | 4.3368          | 39.8649 | 11.9824 | 24.5459 | 36.3921   | 113.65  |
| 3.9558        | 169.57 | 7800 | 4.3363          | 39.8428 | 12.0892 | 24.0175 | 36.67     | 112.7   |
| 3.928         | 171.74 | 7900 | 4.3364          | 39.2281 | 11.8456 | 23.7212 | 36.2005   | 113.95  |
| 3.9351        | 173.91 | 8000 | 4.3363          | 39.9798 | 12.4387 | 23.7687 | 36.6472   | 115.45  |
| 3.9326        | 176.09 | 8100 | 4.3363          | 39.9772 | 12.1193 | 24.1518 | 36.5791   | 117.4   |
| 3.9387        | 178.26 | 8200 | 4.3363          | 39.8629 | 12.1719 | 23.9446 | 36.345    | 115.075 |
| 3.9204        | 180.43 | 8300 | 4.3358          | 39.9738 | 12.3072 | 23.8641 | 36.4802   | 116.3   |
| 3.9418        | 182.61 | 8400 | 4.3357          | 40.1451 | 12.4144 | 24.1553 | 36.4251   | 116.025 |
| 3.9289        | 184.78 | 8500 | 4.3357          | 39.7241 | 12.0543 | 24.0752 | 36.0847   | 115.8   |
| 3.9176        | 186.96 | 8600 | 4.3358          | 39.7969 | 12.0967 | 24.123  | 36.2664   | 118.6   |
| 3.9097        | 189.13 | 8700 | 4.3356          | 39.4096 | 11.9872 | 24.0609 | 35.8662   | 117.2   |
| 3.938         | 191.3  | 8800 | 4.3354          | 39.4695 | 11.9343 | 24.0295 | 35.9372   | 117.025 |
| 3.9239        | 193.48 | 8900 | 4.3352          | 39.3231 | 12.0965 | 23.9131 | 35.9555   | 117.275 |
| 3.91          | 195.65 | 9000 | 4.3354          | 39.5932 | 12.1808 | 23.9233 | 36.0864   | 116.925 |
| 3.9234        | 197.83 | 9100 | 4.3354          | 39.5539 | 12.1134 | 23.9163 | 36.0299   | 117.225 |
| 3.9263        | 200.0  | 9200 | 4.3354          | 39.5539 | 12.1134 | 23.9163 | 36.0299   | 117.225 |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0+cu113
- Datasets 2.1.0
- Tokenizers 0.12.1
