pytorch tensorboard bart text2text-generation transformers generated_from_trainer license:mit model-index autotrain_compatible
---
license: mit
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: bart-cnn-pubmed-arxiv-pubmed-v3-e64
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bart-cnn-pubmed-arxiv-pubmed-v3-e64

This model is a fine-tuned version of [theojolliffe/bart-cnn-pubmed-arxiv-pubmed](https://huggingface.co/theojolliffe/bart-cnn-pubmed-arxiv-pubmed) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0630
- Rouge1: 58.7
- Rouge2: 47.8042
- Rougel: 50.6967
- Rougelsum: 57.5543
- Gen Len: 142.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 64
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len  |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:--------:|
| No log        | 1.0   | 398   | 0.9499          | 53.8396 | 34.0954 | 35.6734 | 51.3453   | 142.0    |
| 1.1219        | 2.0   | 796   | 0.8223          | 53.0414 | 33.3193 | 35.7448 | 50.1675   | 142.0    |
| 0.6681        | 3.0   | 1194  | 0.7689          | 53.6684 | 35.3651 | 37.7087 | 51.1441   | 142.0    |
| 0.4393        | 4.0   | 1592  | 0.7694          | 53.9066 | 35.3925 | 38.8917 | 51.6172   | 142.0    |
| 0.4393        | 5.0   | 1990  | 0.7597          | 54.0746 | 36.1026 | 39.1318 | 51.9272   | 142.0    |
| 0.2947        | 6.0   | 2388  | 0.8284          | 53.1168 | 34.7428 | 38.0573 | 50.9563   | 142.0    |
| 0.2016        | 7.0   | 2786  | 0.7951          | 55.7222 | 39.0458 | 42.5265 | 53.5359   | 142.0    |
| 0.1422        | 8.0   | 3184  | 0.7793          | 56.2376 | 40.3348 | 43.435  | 54.3228   | 142.0    |
| 0.1096        | 9.0   | 3582  | 0.8260          | 55.0372 | 39.0552 | 42.5403 | 53.0694   | 142.0    |
| 0.1096        | 10.0  | 3980  | 0.8397          | 53.849  | 37.519  | 40.674  | 52.1357   | 141.7037 |
| 0.0881        | 11.0  | 4378  | 0.8504          | 56.4835 | 41.0484 | 44.9407 | 54.3557   | 142.0    |
| 0.0693        | 12.0  | 4776  | 0.8285          | 55.7705 | 39.8585 | 43.722  | 53.7607   | 142.0    |
| 0.0572        | 13.0  | 5174  | 0.8327          | 57.932  | 43.5378 | 46.8233 | 55.8739   | 142.0    |
| 0.0461        | 14.0  | 5572  | 0.8720          | 57.6733 | 42.9742 | 45.8698 | 56.018    | 142.0    |
| 0.0461        | 15.0  | 5970  | 0.8723          | 57.6072 | 42.6946 | 45.2551 | 55.8486   | 142.0    |
| 0.0416        | 16.0  | 6368  | 0.8764          | 57.1973 | 43.1931 | 46.4492 | 55.3842   | 142.0    |
| 0.0343        | 17.0  | 6766  | 0.8638          | 57.4474 | 43.3544 | 46.3026 | 55.7863   | 142.0    |
| 0.03          | 18.0  | 7164  | 0.9234          | 57.9166 | 43.8551 | 46.6473 | 56.3895   | 142.0    |
| 0.0252        | 19.0  | 7562  | 0.9393          | 58.2908 | 45.2321 | 47.1398 | 56.6618   | 142.0    |
| 0.0252        | 20.0  | 7960  | 0.8966          | 59.2798 | 46.381  | 49.3514 | 57.6061   | 142.0    |
| 0.024         | 21.0  | 8358  | 0.9056          | 57.8409 | 44.2048 | 47.3329 | 56.2568   | 142.0    |
| 0.0195        | 22.0  | 8756  | 0.9424          | 57.551  | 44.6847 | 47.2771 | 56.2391   | 142.0    |
| 0.0182        | 23.0  | 9154  | 0.9361          | 59.1078 | 46.4704 | 49.4178 | 57.6796   | 142.0    |
| 0.0169        | 24.0  | 9552  | 0.9456          | 56.7966 | 43.3135 | 46.4208 | 55.4646   | 142.0    |
| 0.0169        | 25.0  | 9950  | 0.9867          | 59.5561 | 47.4638 | 50.0725 | 58.2388   | 141.8519 |
| 0.0147        | 26.0  | 10348 | 0.9727          | 58.2574 | 44.9904 | 47.2701 | 56.4274   | 142.0    |
| 0.0125        | 27.0  | 10746 | 0.9589          | 58.6792 | 45.8465 | 48.0781 | 57.0755   | 142.0    |
| 0.0117        | 28.0  | 11144 | 0.9635          | 59.1118 | 46.6614 | 50.0552 | 57.6153   | 142.0    |
| 0.0103        | 29.0  | 11542 | 0.9623          | 58.2517 | 45.6401 | 48.5888 | 56.7733   | 142.0    |
| 0.0103        | 30.0  | 11940 | 0.9752          | 59.0707 | 47.203  | 49.7992 | 57.6216   | 142.0    |
| 0.0096        | 31.0  | 12338 | 0.9610          | 57.6781 | 44.0504 | 47.6718 | 56.1201   | 142.0    |
| 0.0089        | 32.0  | 12736 | 0.9705          | 58.5592 | 45.7397 | 48.681  | 57.0302   | 142.0    |
| 0.008         | 33.0  | 13134 | 0.9989          | 58.1997 | 45.6345 | 48.2551 | 56.8571   | 141.7778 |
| 0.0075        | 34.0  | 13532 | 0.9880          | 57.9632 | 44.7845 | 47.8763 | 56.3979   | 142.0    |
| 0.0075        | 35.0  | 13930 | 1.0041          | 58.1316 | 46.2737 | 49.5986 | 56.8263   | 142.0    |
| 0.0061        | 36.0  | 14328 | 0.9923          | 58.4686 | 46.1735 | 49.1299 | 57.0331   | 142.0    |
| 0.0066        | 37.0  | 14726 | 1.0157          | 58.4277 | 45.6559 | 49.1739 | 56.8198   | 141.6481 |
| 0.0052        | 38.0  | 15124 | 1.0220          | 58.5166 | 46.3883 | 50.0964 | 57.0104   | 142.0    |
| 0.0049        | 39.0  | 15522 | 0.9949          | 59.3697 | 47.0609 | 50.2733 | 58.1388   | 142.0    |
| 0.0049        | 40.0  | 15920 | 1.0368          | 59.9537 | 48.4059 | 51.8185 | 58.8002   | 142.0    |
| 0.0039        | 41.0  | 16318 | 1.0228          | 58.2093 | 46.4807 | 49.54   | 56.9994   | 142.0    |
| 0.0041        | 42.0  | 16716 | 1.0218          | 57.6376 | 45.4951 | 49.003  | 56.4606   | 142.0    |
| 0.0035        | 43.0  | 17114 | 1.0381          | 57.2845 | 43.9593 | 46.779  | 55.6106   | 142.0    |
| 0.0059        | 44.0  | 17512 | 1.0316          | 58.5506 | 46.2111 | 49.4844 | 56.9506   | 142.0    |
| 0.0059        | 45.0  | 17910 | 1.0388          | 58.8383 | 47.6053 | 50.6187 | 57.7125   | 142.0    |
| 0.0028        | 46.0  | 18308 | 1.0068          | 59.3198 | 47.6888 | 50.2478 | 58.0      | 142.0    |
| 0.0028        | 47.0  | 18706 | 1.0446          | 58.8938 | 46.7524 | 49.5642 | 57.3659   | 142.0    |
| 0.0022        | 48.0  | 19104 | 1.0347          | 59.8253 | 48.3871 | 51.3949 | 58.5652   | 142.0    |
| 0.0024        | 49.0  | 19502 | 1.0294          | 60.655  | 50.2339 | 53.1662 | 59.3333   | 142.0    |
| 0.0024        | 50.0  | 19900 | 1.0225          | 58.5131 | 47.3009 | 50.1642 | 57.2287   | 142.0    |
| 0.0022        | 51.0  | 20298 | 1.0320          | 59.6101 | 47.4104 | 50.5291 | 58.075    | 142.0    |
| 0.0018        | 52.0  | 20696 | 1.0507          | 58.7957 | 46.8893 | 50.2996 | 57.3662   | 142.0    |
| 0.0015        | 53.0  | 21094 | 1.0599          | 58.9064 | 47.9433 | 51.3082 | 57.6871   | 142.0    |
| 0.0015        | 54.0  | 21492 | 1.0636          | 59.6607 | 48.5737 | 51.2361 | 58.333    | 142.0    |
| 0.0013        | 55.0  | 21890 | 1.0452          | 58.7026 | 46.5286 | 49.9672 | 57.2521   | 142.0    |
| 0.0012        | 56.0  | 22288 | 1.0418          | 58.9452 | 47.7209 | 50.657  | 57.7103   | 142.0    |
| 0.0011        | 57.0  | 22686 | 1.0578          | 58.485  | 46.0691 | 49.811  | 57.2591   | 142.0    |
| 0.0009        | 58.0  | 23084 | 1.0561          | 59.2268 | 48.1987 | 50.1948 | 57.8871   | 142.0    |
| 0.0009        | 59.0  | 23482 | 1.0548          | 59.6307 | 48.1778 | 50.9934 | 58.2098   | 142.0    |
| 0.0009        | 60.0  | 23880 | 1.0498          | 59.5054 | 48.8866 | 51.5977 | 58.1868   | 142.0    |
| 0.0008        | 61.0  | 24278 | 1.0583          | 60.0232 | 49.2518 | 52.2297 | 58.6774   | 142.0    |
| 0.0007        | 62.0  | 24676 | 1.0659          | 59.1755 | 48.4144 | 51.5157 | 58.0416   | 142.0    |
| 0.0007        | 63.0  | 25074 | 1.0622          | 59.1023 | 47.74   | 50.5188 | 57.9707   | 142.0    |
| 0.0007        | 64.0  | 25472 | 1.0630          | 58.7    | 47.8042 | 50.6967 | 57.5543   | 142.0    |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0+cu113
- Datasets 2.1.0
- Tokenizers 0.12.1
