
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# Heem/distilroberta-finetuned-wtner

This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.0055
- Validation Loss: 0.4521
- Train Precision: 0.7410
- Train Recall: 0.8122
- Train F1: 0.775
- Train Accuracy: 0.9382
- Epoch: 69

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 2030, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32

### Training results

| Train Loss | Validation Loss | Train Precision | Train Recall | Train F1 | Train Accuracy | Epoch |
|:----------:|:---------------:|:---------------:|:------------:|:--------:|:--------------:|:-----:|
| 1.3579     | 0.8909          | 0.0             | 0.0          | 0.0      | 0.7744         | 0     |
| 0.7332     | 0.6231          | 0.3526          | 0.2926       | 0.3198   | 0.8256         | 1     |
| 0.5037     | 0.4471          | 0.3927          | 0.3755       | 0.3839   | 0.8575         | 2     |
| 0.3675     | 0.3776          | 0.484           | 0.5284       | 0.5052   | 0.8855         | 3     |
| 0.2890     | 0.3519          | 0.5149          | 0.6026       | 0.5553   | 0.9039         | 4     |
| 0.2367     | 0.3317          | 0.5820          | 0.6507       | 0.6144   | 0.9150         | 5     |
| 0.1942     | 0.2970          | 0.6220          | 0.6900       | 0.6542   | 0.9237         | 6     |
| 0.1599     | 0.3040          | 0.6375          | 0.6681       | 0.6525   | 0.9217         | 7     |
| 0.1281     | 0.3037          | 0.6774          | 0.7336       | 0.7044   | 0.9304         | 8     |
| 0.1097     | 0.3127          | 0.708           | 0.7729       | 0.7390   | 0.9309         | 9     |
| 0.0915     | 0.3114          | 0.6836          | 0.7642       | 0.7216   | 0.9290         | 10    |
| 0.0765     | 0.3190          | 0.7072          | 0.8122       | 0.7561   | 0.9372         | 11    |
| 0.0665     | 0.3169          | 0.7154          | 0.7904       | 0.7510   | 0.9353         | 12    |
| 0.0543     | 0.3251          | 0.7059          | 0.7860       | 0.7438   | 0.9329         | 13    |
| 0.0472     | 0.3307          | 0.7181          | 0.8122       | 0.7623   | 0.9357         | 14    |
| 0.0427     | 0.3639          | 0.7148          | 0.7991       | 0.7546   | 0.9357         | 15    |
| 0.0380     | 0.3373          | 0.7373          | 0.8210       | 0.7769   | 0.9377         | 16    |
| 0.0380     | 0.3422          | 0.7449          | 0.8035       | 0.7731   | 0.9372         | 17    |
| 0.0304     | 0.3455          | 0.7530          | 0.8122       | 0.7815   | 0.9386         | 18    |
| 0.0271     | 0.3584          | 0.7294          | 0.8122       | 0.7686   | 0.9377         | 19    |
| 0.0249     | 0.3661          | 0.7291          | 0.7991       | 0.7625   | 0.9377         | 20    |
| 0.0205     | 0.3683          | 0.7352          | 0.8122       | 0.7718   | 0.9391         | 21    |
| 0.0212     | 0.3855          | 0.7331          | 0.8035       | 0.7667   | 0.9382         | 22    |
| 0.0188     | 0.3814          | 0.7419          | 0.8035       | 0.7715   | 0.9391         | 23    |
| 0.0189     | 0.3889          | 0.7352          | 0.8122       | 0.7718   | 0.9357         | 24    |
| 0.0161     | 0.3913          | 0.7379          | 0.7991       | 0.7673   | 0.9382         | 25    |
| 0.0154     | 0.3872          | 0.7470          | 0.8122       | 0.7782   | 0.9406         | 26    |
| 0.0144     | 0.3934          | 0.7326          | 0.8253       | 0.7762   | 0.9401         | 27    |
| 0.0154     | 0.4167          | 0.7255          | 0.8079       | 0.7645   | 0.9343         | 28    |
| 0.0135     | 0.3976          | 0.7341          | 0.8079       | 0.7692   | 0.9362         | 29    |
| 0.0119     | 0.4118          | 0.7510          | 0.8297       | 0.7884   | 0.9382         | 30    |
| 0.0103     | 0.4112          | 0.7323          | 0.8122       | 0.7702   | 0.9372         | 31    |
| 0.0103     | 0.4172          | 0.7362          | 0.8166       | 0.7743   | 0.9382         | 32    |
| 0.0111     | 0.4157          | 0.7283          | 0.8079       | 0.7660   | 0.9382         | 33    |
| 0.0103     | 0.4152          | 0.7262          | 0.7991       | 0.7609   | 0.9372         | 34    |
| 0.0117     | 0.4090          | 0.7188          | 0.8035       | 0.7588   | 0.9377         | 35    |
| 0.0098     | 0.4268          | 0.7302          | 0.8035       | 0.7651   | 0.9367         | 36    |
| 0.0082     | 0.4354          | 0.7233          | 0.7991       | 0.7593   | 0.9362         | 37    |
| 0.0096     | 0.4298          | 0.7154          | 0.7904       | 0.7510   | 0.9357         | 38    |
| 0.0093     | 0.4294          | 0.7273          | 0.8035       | 0.7635   | 0.9362         | 39    |
| 0.0084     | 0.4266          | 0.7298          | 0.7904       | 0.7589   | 0.9348         | 40    |
| 0.0076     | 0.4230          | 0.7251          | 0.7948       | 0.7583   | 0.9357         | 41    |
| 0.0068     | 0.4243          | 0.7075          | 0.7817       | 0.7427   | 0.9329         | 42    |
| 0.0080     | 0.4379          | 0.7137          | 0.7729       | 0.7421   | 0.9338         | 43    |
| 0.0067     | 0.4361          | 0.7302          | 0.8035       | 0.7651   | 0.9362         | 44    |
| 0.0066     | 0.4377          | 0.7341          | 0.8079       | 0.7692   | 0.9367         | 45    |
| 0.0056     | 0.4357          | 0.7222          | 0.7948       | 0.7568   | 0.9362         | 46    |
| 0.0060     | 0.4393          | 0.7205          | 0.7991       | 0.7578   | 0.9362         | 47    |
| 0.0060     | 0.4429          | 0.7194          | 0.7948       | 0.7552   | 0.9357         | 48    |
| 0.0054     | 0.4416          | 0.7312          | 0.8079       | 0.7676   | 0.9367         | 49    |
| 0.0060     | 0.4413          | 0.7188          | 0.8035       | 0.7588   | 0.9362         | 50    |
| 0.0058     | 0.4381          | 0.7344          | 0.8210       | 0.7753   | 0.9377         | 51    |
| 0.0063     | 0.4388          | 0.7309          | 0.7948       | 0.7615   | 0.9377         | 52    |
| 0.0057     | 0.4402          | 0.7412          | 0.8253       | 0.7810   | 0.9382         | 53    |
| 0.0052     | 0.4381          | 0.7362          | 0.8166       | 0.7743   | 0.9377         | 54    |
| 0.0049     | 0.4407          | 0.7362          | 0.8166       | 0.7743   | 0.9377         | 55    |
| 0.0050     | 0.4394          | 0.7490          | 0.8210       | 0.7833   | 0.9386         | 56    |
| 0.0047     | 0.4481          | 0.7460          | 0.8210       | 0.7817   | 0.9382         | 57    |
| 0.0052     | 0.4544          | 0.748           | 0.8166       | 0.7808   | 0.9367         | 58    |
| 0.0049     | 0.4501          | 0.7430          | 0.8079       | 0.7741   | 0.9362         | 59    |
| 0.0050     | 0.4504          | 0.744           | 0.8122       | 0.7766   | 0.9367         | 60    |
| 0.0047     | 0.4517          | 0.7312          | 0.8079       | 0.7676   | 0.9372         | 61    |
| 0.0049     | 0.4526          | 0.7450          | 0.8166       | 0.7792   | 0.9382         | 62    |
| 0.0049     | 0.4534          | 0.7490          | 0.8210       | 0.7833   | 0.9386         | 63    |
| 0.0056     | 0.4543          | 0.748           | 0.8166       | 0.7808   | 0.9386         | 64    |
| 0.0044     | 0.4522          | 0.7410          | 0.8122       | 0.775    | 0.9382         | 65    |
| 0.0047     | 0.4522          | 0.7410          | 0.8122       | 0.775    | 0.9382         | 66    |
| 0.0050     | 0.4521          | 0.7410          | 0.8122       | 0.775    | 0.9382         | 67    |
| 0.0049     | 0.4521          | 0.7410          | 0.8122       | 0.775    | 0.9382         | 68    |
| 0.0055     | 0.4521          | 0.7410          | 0.8122       | 0.775    | 0.9382         | 69    |


### Framework versions

- Transformers 4.20.1
- TensorFlow 2.9.1
- Datasets 2.4.0
- Tokenizers 0.12.1

{"license": "apache-2.0", "tags": ["generated_from_keras_callback"], "model-index": [{"name": "Heem/distilroberta-finetuned-wtner", "results": []}]}
generated_from_keras_callback license:apache-2.0 region:us
TAGS
#generated_from_keras_callback #license-apache-2.0 #region-us 

Heem/distilroberta-finetuned-wtner
==================================


This model is a fine-tuned version of distilroberta-base on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.0055
* Validation Loss: 0.4521
* Train Precision: 0.7410
* Train Recall: 0.8122
* Train F1: 0.775
* Train Accuracy: 0.9382
* Epoch: 69


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2030, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.20.1
* TensorFlow 2.9.1
* Datasets 2.4.0
* Tokenizers 0.12.1



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2030, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.20.1
* TensorFlow 2.9.1
* Datasets 2.4.0
* Tokenizers 0.12.1
TAGS
#generated_from_keras_callback #license-apache-2.0 #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2030, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.20.1
* TensorFlow 2.9.1
* Datasets 2.4.0
* Tokenizers 0.12.1
passage: TAGS
#generated_from_keras_callback #license-apache-2.0 #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': {'class\_name': 'PolynomialDecay', 'config': {'initial\_learning\_rate': 2e-05, 'decay\_steps': 2030, 'end\_learning\_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32### Training results### Framework versions


* Transformers 4.20.1
* TensorFlow 2.9.1
* Datasets 2.4.0
* Tokenizers 0.12.1