File size: 4,837 Bytes
5b3e33d 79c6b95 928151f f0c0674 5b3e33d f0c0674 5b3e33d c15ffb5 5b3e33d 1f0275a 5b3e33d 8d103e8 5b3e33d 1f0275a 5b3e33d 5373dc3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
---
language:
- it
tags:
- summarization
datasets:
- ARTeLab/mlsum-it
metrics:
- rouge
base_model: gsarti/it5-base
model-index:
- name: summarization_mlsum
results:
- task:
type: summarization
name: Summarization
dataset:
name: cnn_dailymail
type: cnn_dailymail
config: 3.0.0
split: test
metrics:
- type: rouge
value: 10.775
name: ROUGE-1
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2MxZWFhOTMxYTBmMjVhZmE0MzU5NDQzMGVlYWU5MjJmYjBiMGI5Y2U1ZmMwZDQzYWUwNDEwNDI1ZjY2ODA5MyIsInZlcnNpb24iOjF9.EqOHhgehPD-OVAK2vCdFndZlhiyh3-Vc89D_ujisSgK-shrouep7JhKV4hYtp-m5PbEvAQSk8PWJsYBlwV00Bw
- type: rouge
value: 3.0633
name: ROUGE-2
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjNiZGNlNjEyOTI4YjQ2OWIzNjczNjU5ZWUyOTk3MTNhMDY0ZmI5ZDcyOGEwOTVjNjFmYjJiMjBhZDBiY2Y1YiIsInZlcnNpb24iOjF9.Hznn3spvnWWUpR4KVQ20UP-rM2MFDtRCCjtaiUxRRvC_46KpsPoyKme2h_X3QFW-xKPMLj4BLaJOLRPTrXO0Dw
- type: rouge
value: 9.2018
name: ROUGE-L
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNzVhOGUwYjYxYzQwYTVhZDUzYzIwMjc1ZWE5ZjQ3ZWRkNzlmZGY1MmY5ODk3ZGNiY2Q4YjJiOWY3OTJjOWU4NyIsInZlcnNpb24iOjF9.yKGKdtzMlv81Ym7bCPlEjDMrWhwKO7GHuog9I5PjvnmOwtM2TVTLk8XqIvU3_GnlSBfffNEe12pCJ-zQ27Q3BQ
- type: rouge
value: 10.1469
name: ROUGE-LSUM
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWExM2I0ZjhmYjE5YzQ1YjhiMTdmOTQzNDc5ODM0NjQ5MzMyNDVkMTUxMzg0MDVhNDU2OTNlN2EzNTc1ZGFlYyIsInZlcnNpb24iOjF9.-6_6JVdsBYdDN9Gi5iuEPchyaY0K3az06nTylQKA22bX1mBziQ2Y4z8crdzxF_hf_z1pPunWnhyLj3yUn4KPCA
- type: loss
value: 4.3302483558654785
name: loss
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWI4OWNlYTJiNGU5OWRhYmE4MTFiY2YxNTk4NWY2M2RjMDE2ODY4NzBmYWVmNjUxYmY2MzZiNjBmMTY1ODgxMiIsInZlcnNpb24iOjF9.QDPl5BGgPsu9XdEC0TA_Zjhb47nEHFM9ysBTvDs75-1kp_Y6aqB-xIPFp03llsXBHGnbyAr4WQhFRtxDdlrKDw
- type: gen_len
value: 18.9984
name: gen_len
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODYxZGRhZWFiYzQwODlhNzM0NTlkNzVhN2FkOWYzZDczMmE1MjU0ODdhNGRmYjc1NTE0NTIyYjhhZjRhOGQ1YyIsInZlcnNpb24iOjF9.z5jkmfzM0btlSKulTh7w6FZk66q0JRS803mVgBT1nL88vDaCTezb4wRYbXazssdfo2V8J-EY3r_VwVTAxQxgAw
---
# summarization_mlsum
This model is a fine-tuned version of [gsarti/it5-base](https://huggingface.co/gsarti/it5-base) on MLSum-it for Abstractive Summarization.
It achieves the following results:
- Loss: 2.0190
- Rouge1: 19.3739
- Rouge2: 5.9753
- Rougel: 16.691
- Rougelsum: 16.7862
- Gen Len: 32.5268
## Usage
```python
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("ARTeLab/it5-summarization-mlsum")
model = T5ForConditionalGeneration.from_pretrained("ARTeLab/it5-summarization-mlsum")
```
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 6
- eval_batch_size: 6
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4.0
### Framework versions
- Transformers 4.12.0.dev0
- Pytorch 1.9.1+cu102
- Datasets 1.12.1
- Tokenizers 0.10.3
# Citation
More details and results in [published work](https://www.mdpi.com/2078-2489/13/5/228)
```
@Article{info13050228,
AUTHOR = {Landro, Nicola and Gallo, Ignazio and La Grassa, Riccardo and Federici, Edoardo},
TITLE = {Two New Datasets for Italian-Language Abstractive Text Summarization},
JOURNAL = {Information},
VOLUME = {13},
YEAR = {2022},
NUMBER = {5},
ARTICLE-NUMBER = {228},
URL = {https://www.mdpi.com/2078-2489/13/5/228},
ISSN = {2078-2489},
ABSTRACT = {Text summarization aims to produce a short summary containing relevant parts from a given text. Due to the lack of data for abstractive summarization on low-resource languages such as Italian, we propose two new original datasets collected from two Italian news websites with multi-sentence summaries and corresponding articles, and from a dataset obtained by machine translation of a Spanish summarization dataset. These two datasets are currently the only two available in Italian for this task. To evaluate the quality of these two datasets, we used them to train a T5-base model and an mBART model, obtaining good results with both. To better evaluate the results obtained, we also compared the same models trained on automatically translated datasets, and the resulting summaries in the same training language, with the automatically translated summaries, which demonstrated the superiority of the models obtained from the proposed datasets.},
DOI = {10.3390/info13050228}
}
``` |