File size: 1,232 Bytes
93f197e ffdb6c4 93f197e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
---
license: apache-2.0
datasets:
- ambrosfitz/cnn-daily-grammar
language:
- en
base_model:
- google-t5/t5-base
pipeline_tag: summarization
---
# T5-CNN-Grammar-Enhanced
## Model Description
A T5-base model fine-tuned on the CNN Daily Grammar dataset for enhanced summarization with grammatical structure awareness.
## Usage
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqGeneration
tokenizer = AutoTokenizer.from_pretrained("username/t5-cnn-grammar-enhanced")
model = AutoModelForSeq2SeqGeneration.from_pretrained("username/t5-cnn-grammar-enhanced")
```
## Training Details
- Base model: t5-base
- Dataset: CNN Daily Grammar
- Training type: Fine-tuning
- Framework: PyTorch
- Epochs: 10
- Batch size: 8
- Learning rate: 2e-5
- Loss: Focal Loss
- Scheduler: Linear warmup
- Best validation loss: 0.7759
## Model Architecture
- Encoder-decoder transformer
- Grammar-enhanced input structure
- Focal loss for detail retention
## Evaluation Results
Final validation metrics:
- Loss: 0.7759
- Strong performance on detail retention and factual accuracy
## Limitations
- Limited to news article summarization
- May omit specific numerical details
- Best suited for formal news content
## License
Apache 2.0 |