|
--- |
|
license: mit |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: deberta-v3-large |
|
results: [] |
|
--- |
|
|
|
# deberta-v3-large-sentiment |
|
|
|
This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on an [tweet_eval](https://huggingface.co/datasets/tweet_eval) dataset. |
|
|
|
## Model description |
|
|
|
Test set results: |
|
|
|
| Model | Emotion | Hate | Irony | Offensive | Sentiment | |
|
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | |
|
| deberta-v3-large | **86.3** | **61.3** | **87.1** | **86.4** | **73.9** | |
|
| BERTweet | 79.3 | - | 82.1 | 79.5 | 73.4 | |
|
| RoB-RT | 79.5 | 52.3 | 61.7 | 80.5 | 69.3 | |
|
|
|
[source:papers_with_code](https://paperswithcode.com/sota/sentiment-analysis-on-tweeteval) |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
Classifying attributes of interest on tweeter like data. |
|
|
|
## Training and evaluation data |
|
|
|
[tweet_eval](https://huggingface.co/datasets/tweet_eval) dataset. |
|
|
|
## Training procedure |
|
|
|
Fine tuned and evaluated with [run_glue.py]() |
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 7e-06 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 2 |
|
- total_train_batch_size: 32 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 50 |
|
- num_epochs: 10.0 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
| 0.6417 | 0.27 | 100 | 0.6283 | 0.6533 | |
|
| 0.5105 | 0.54 | 200 | 0.4588 | 0.7915 | |
|
| 0.4554 | 0.81 | 300 | 0.4500 | 0.7968 | |
|
| 0.4212 | 1.08 | 400 | 0.4773 | 0.7938 | |
|
| 0.4054 | 1.34 | 500 | 0.4311 | 0.7983 | |
|
| 0.3922 | 1.61 | 600 | 0.4588 | 0.7998 | |
|
| 0.3776 | 1.88 | 700 | 0.4367 | 0.8066 | |
|
| 0.3535 | 2.15 | 800 | 0.4675 | 0.8074 | |
|
| 0.33 | 2.42 | 900 | 0.4874 | 0.8021 | |
|
| 0.3113 | 2.69 | 1000 | 0.4949 | 0.8044 | |
|
| 0.3203 | 2.96 | 1100 | 0.4550 | 0.8059 | |
|
| 0.248 | 3.23 | 1200 | 0.4858 | 0.8036 | |
|
| 0.2478 | 3.49 | 1300 | 0.5299 | 0.8029 | |
|
| 0.2371 | 3.76 | 1400 | 0.5013 | 0.7991 | |
|
| 0.2388 | 4.03 | 1500 | 0.5520 | 0.8021 | |
|
| 0.1744 | 4.3 | 1600 | 0.6687 | 0.7915 | |
|
| 0.1788 | 4.57 | 1700 | 0.7560 | 0.7689 | |
|
| 0.1652 | 4.84 | 1800 | 0.6985 | 0.7832 | |
|
| 0.1596 | 5.11 | 1900 | 0.7191 | 0.7915 | |
|
| 0.1214 | 5.38 | 2000 | 0.9097 | 0.7893 | |
|
| 0.1432 | 5.64 | 2100 | 0.9184 | 0.7787 | |
|
| 0.1145 | 5.91 | 2200 | 0.9620 | 0.7878 | |
|
| 0.1069 | 6.18 | 2300 | 0.9489 | 0.7893 | |
|
| 0.1012 | 6.45 | 2400 | 1.0107 | 0.7817 | |
|
| 0.0942 | 6.72 | 2500 | 1.0021 | 0.7885 | |
|
| 0.087 | 6.99 | 2600 | 1.1090 | 0.7915 | |
|
| 0.0598 | 7.26 | 2700 | 1.1735 | 0.7795 | |
|
| 0.0742 | 7.53 | 2800 | 1.1433 | 0.7817 | |
|
| 0.073 | 7.79 | 2900 | 1.1343 | 0.7953 | |
|
| 0.0553 | 8.06 | 3000 | 1.2258 | 0.7840 | |
|
| 0.0474 | 8.33 | 3100 | 1.2461 | 0.7817 | |
|
| 0.0515 | 8.6 | 3200 | 1.2996 | 0.7825 | |
|
| 0.0551 | 8.87 | 3300 | 1.2819 | 0.7855 | |
|
| 0.0541 | 9.14 | 3400 | 1.2808 | 0.7855 | |
|
| 0.0465 | 9.41 | 3500 | 1.3398 | 0.7817 | |
|
| 0.0407 | 9.68 | 3600 | 1.3231 | 0.7825 | |
|
| 0.0343 | 9.94 | 3700 | 1.3330 | 0.7825 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.20.0.dev0 |
|
- Pytorch 1.9.0 |
|
- Datasets 2.2.2 |
|
- Tokenizers 0.11.6 |
|
|
