
# Clickbait1

This model is a fine-tuned version of [microsoft/Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384) on the [Webis-Clickbait-17](https://zenodo.org/record/5530410) dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0257

## Model description

MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers".

We fine tune this model to evaluate (regression) the clickbait level of title news.

## Intended uses & limitations

Model looks like the model described in the paper [Predicting Clickbait Strength in Online Social Media](https://aclanthology.org/2020.coling-main.425/) by Indurthi Vijayasaradhi, Syed Bakhtiyar, Gupta Manish, Varma Vasudeva.

The model was trained with english titles.

## Training and evaluation data

We trained the model with the official training data for the chalenge (clickbait17-train-170630.zip (894 MiB, 19538 posts), plus another set that was just available after the end of the challenge (clickbait17-train-170331.zip (157 MiB, 2459 posts).

## Training procedure
Code can be find in [Github](https://github.com/caush/Clickbait).

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 0.05  | 50   | 0.0571          |
| No log        | 0.09  | 100  | 0.0448          |
| No log        | 0.14  | 150  | 0.0391          |
| No log        | 0.18  | 200  | 0.0326          |
| No log        | 0.23  | 250  | 0.0343          |
| No log        | 0.27  | 300  | 0.0343          |
| No log        | 0.32  | 350  | 0.0343          |
| No log        | 0.36  | 400  | 0.0346          |
| No log        | 0.41  | 450  | 0.0343          |
| 0.0388        | 0.46  | 500  | 0.0297          |
| 0.0388        | 0.5   | 550  | 0.0293          |
| 0.0388        | 0.55  | 600  | 0.0301          |
| 0.0388        | 0.59  | 650  | 0.0290          |
| 0.0388        | 0.64  | 700  | 0.0326          |
| 0.0388        | 0.68  | 750  | 0.0285          |
| 0.0388        | 0.73  | 800  | 0.0285          |
| 0.0388        | 0.77  | 850  | 0.0275          |
| 0.0388        | 0.82  | 900  | 0.0314          |
| 0.0388        | 0.87  | 950  | 0.0309          |
| 0.0297        | 0.91  | 1000 | 0.0277          |
| 0.0297        | 0.96  | 1050 | 0.0281          |
| 0.0297        | 1.0   | 1100 | 0.0273          |
| 0.0297        | 1.05  | 1150 | 0.0270          |
| 0.0297        | 1.09  | 1200 | 0.0291          |
| 0.0297        | 1.14  | 1250 | 0.0293          |
| 0.0297        | 1.18  | 1300 | 0.0269          |
| 0.0297        | 1.23  | 1350 | 0.0276          |
| 0.0297        | 1.28  | 1400 | 0.0279          |
| 0.0297        | 1.32  | 1450 | 0.0267          |
| 0.0265        | 1.37  | 1500 | 0.0270          |
| 0.0265        | 1.41  | 1550 | 0.0300          |
| 0.0265        | 1.46  | 1600 | 0.0274          |
| 0.0265        | 1.5   | 1650 | 0.0274          |
| 0.0265        | 1.55  | 1700 | 0.0266          |
| 0.0265        | 1.59  | 1750 | 0.0267          |
| 0.0265        | 1.64  | 1800 | 0.0267          |
| 0.0265        | 1.68  | 1850 | 0.0280          |
| 0.0265        | 1.73  | 1900 | 0.0274          |
| 0.0265        | 1.78  | 1950 | 0.0272          |
| 0.025         | 1.82  | 2000 | 0.0261          |
| 0.025         | 1.87  | 2050 | 0.0268          |
| 0.025         | 1.91  | 2100 | 0.0268          |
| 0.025         | 1.96  | 2150 | 0.0259          |
| 0.025         | 2.0   | 2200 | 0.0257          |
| 0.025         | 2.05  | 2250 | 0.0260          |
| 0.025         | 2.09  | 2300 | 0.0263          |
| 0.025         | 2.14  | 2350 | 0.0262          |
| 0.025         | 2.19  | 2400 | 0.0269          |
| 0.025         | 2.23  | 2450 | 0.0262          |
| 0.0223        | 2.28  | 2500 | 0.0262          |
| 0.0223        | 2.32  | 2550 | 0.0267          |
| 0.0223        | 2.37  | 2600 | 0.0260          |
| 0.0223        | 2.41  | 2650 | 0.0260          |
| 0.0223        | 2.46  | 2700 | 0.0259          |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0a0+17540c5
- Datasets 2.1.0
- Tokenizers 0.12.1

{"license": "mit", "tags": ["generated_from_trainer"], "model-index": [{"name": "Clickbait1", "results": []}]}
transformers pytorch tensorboard bert text-classification generated_from_trainer license:mit autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #bert #text-classification #generated_from_trainer #license-mit #autotrain_compatible #endpoints_compatible #region-us 

Clickbait1
==========


This model is a fine-tuned version of microsoft/Multilingual-MiniLM-L12-H384 on the Webis-Clickbait-17 dataset.
It achieves the following results on the evaluation set:


* Loss: 0.0257


Model description
-----------------


MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers".


We fine tune this model to evaluate (regression) the clickbait level of title news.


Intended uses & limitations
---------------------------


Model looks like the model described in the paper Predicting Clickbait Strength in Online Social Media by Indurthi Vijayasaradhi, Syed Bakhtiyar, Gupta Manish, Varma Vasudeva.


The model was trained with english titles.


Training and evaluation data
----------------------------


We trained the model with the official training data for the chalenge (URL (894 MiB, 19538 posts), plus another set that was just available after the end of the challenge (URL (157 MiB, 2459 posts).


Training procedure
------------------


Code can be find in Github.


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 4


### Training results



### Framework versions


* Transformers 4.18.0
* Pytorch 1.11.0a0+17540c5
* Datasets 2.1.0
* Tokenizers 0.12.1



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 4 ### Training results ### Framework versions


* Transformers 4.18.0
* Pytorch 1.11.0a0+17540c5
* Datasets 2.1.0
* Tokenizers 0.12.1
TAGS
#transformers #pytorch #tensorboard #bert #text-classification #generated_from_trainer #license-mit #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 4 ### Training results ### Framework versions


* Transformers 4.18.0
* Pytorch 1.11.0a0+17540c5
* Datasets 2.1.0
* Tokenizers 0.12.1
passage: TAGS
#transformers #pytorch #tensorboard #bert #text-classification #generated_from_trainer #license-mit #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 4### Training results### Framework versions


* Transformers 4.18.0
* Pytorch 1.11.0a0+17540c5
* Datasets 2.1.0
* Tokenizers 0.12.1