pytorch tensorboard distilbert text-classification transformers generated_from_trainer license:apache-2.0 model-index
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: DEREXP
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# DEREXP

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.1590
- Mse: 3.1590
- Mae: 1.3397
- R2: 0.4465
- Accuracy: 0.2528

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Mse    | Mae    | R2     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:--------:|
| 14.7557       | 0.01  | 500   | 4.3307          | 4.3307 | 1.6240 | 0.2411 | 0.1976   |
| 4.5754        | 0.02  | 1000  | 4.1273          | 4.1273 | 1.5719 | 0.2768 | 0.2084   |
| 4.2925        | 0.02  | 1500  | 4.3074          | 4.3074 | 1.6155 | 0.2452 | 0.2012   |
| 3.9816        | 0.03  | 2000  | 3.7767          | 3.7767 | 1.5008 | 0.3382 | 0.2134   |
| 3.9171        | 0.04  | 2500  | 3.7033          | 3.7033 | 1.4732 | 0.3511 | 0.2304   |
| 3.946         | 0.05  | 3000  | 3.6217          | 3.6217 | 1.4552 | 0.3654 | 0.2352   |
| 4.1           | 0.06  | 3500  | 3.6101          | 3.6101 | 1.4612 | 0.3674 | 0.2216   |
| 3.8535        | 0.06  | 4000  | 3.6160          | 3.6160 | 1.4576 | 0.3664 | 0.2294   |
| 3.9037        | 0.07  | 4500  | 3.5864          | 3.5864 | 1.4476 | 0.3716 | 0.2374   |
| 3.9358        | 0.08  | 5000  | 3.5087          | 3.5087 | 1.4237 | 0.3852 | 0.2414   |
| 3.8062        | 0.09  | 5500  | 3.6085          | 3.6085 | 1.4595 | 0.3677 | 0.2256   |
| 3.8802        | 0.1   | 6000  | 3.6371          | 3.6371 | 1.4615 | 0.3627 | 0.223    |
| 3.7239        | 0.1   | 6500  | 3.5191          | 3.5191 | 1.4278 | 0.3834 | 0.2324   |
| 3.7618        | 0.11  | 7000  | 3.8408          | 3.8408 | 1.4973 | 0.3270 | 0.2316   |
| 3.7217        | 0.12  | 7500  | 3.8241          | 3.8241 | 1.5046 | 0.3299 | 0.2236   |
| 3.8204        | 0.13  | 8000  | 3.5290          | 3.5290 | 1.4256 | 0.3816 | 0.2388   |
| 3.7211        | 0.14  | 8500  | 3.6903          | 3.6903 | 1.4674 | 0.3534 | 0.227    |
| 3.7243        | 0.14  | 9000  | 3.4718          | 3.4718 | 1.4201 | 0.3917 | 0.231    |
| 3.7713        | 0.15  | 9500  | 3.8970          | 3.8970 | 1.5304 | 0.3171 | 0.2092   |
| 3.6289        | 0.16  | 10000 | 3.5273          | 3.5273 | 1.4255 | 0.3819 | 0.2388   |
| 3.7516        | 0.17  | 10500 | 3.9020          | 3.9020 | 1.5230 | 0.3163 | 0.2138   |
| 3.7491        | 0.18  | 11000 | 3.4809          | 3.4809 | 1.4209 | 0.3901 | 0.2378   |
| 3.7809        | 0.18  | 11500 | 3.8779          | 3.8779 | 1.5087 | 0.3205 | 0.229    |
| 3.7163        | 0.19  | 12000 | 3.5177          | 3.5177 | 1.4330 | 0.3836 | 0.2298   |
| 3.732         | 0.2   | 12500 | 3.9986          | 3.9986 | 1.5401 | 0.2993 | 0.218    |
| 3.7381        | 0.21  | 13000 | 3.4782          | 3.4782 | 1.4277 | 0.3905 | 0.2302   |
| 3.7652        | 0.22  | 13500 | 3.6239          | 3.6239 | 1.4587 | 0.3650 | 0.2244   |
| 3.6003        | 0.22  | 14000 | 3.4873          | 3.4873 | 1.4288 | 0.3889 | 0.2316   |
| 3.6865        | 0.23  | 14500 | 3.5895          | 3.5895 | 1.4511 | 0.3710 | 0.23     |
| 3.7398        | 0.24  | 15000 | 3.8835          | 3.8835 | 1.5183 | 0.3195 | 0.2172   |
| 3.5939        | 0.25  | 15500 | 3.6334          | 3.6334 | 1.4643 | 0.3633 | 0.2256   |
| 3.691         | 0.26  | 16000 | 3.4251          | 3.4251 | 1.3994 | 0.3998 | 0.2488   |
| 3.7279        | 0.26  | 16500 | 3.3956          | 3.3956 | 1.4034 | 0.4050 | 0.2336   |
| 3.797         | 0.27  | 17000 | 3.4029          | 3.4029 | 1.3968 | 0.4037 | 0.2486   |
| 3.684         | 0.28  | 17500 | 3.5831          | 3.5831 | 1.4451 | 0.3721 | 0.2304   |
| 3.5894        | 0.29  | 18000 | 3.6120          | 3.6120 | 1.4492 | 0.3671 | 0.2338   |
| 3.5938        | 0.3   | 18500 | 3.4975          | 3.4975 | 1.4240 | 0.3871 | 0.231    |
| 3.4948        | 0.3   | 19000 | 3.4791          | 3.4791 | 1.4167 | 0.3904 | 0.24     |
| 3.6527        | 0.31  | 19500 | 3.3409          | 3.3409 | 1.3817 | 0.4146 | 0.2474   |
| 3.5545        | 0.32  | 20000 | 3.3412          | 3.3412 | 1.3860 | 0.4145 | 0.2466   |
| 3.6102        | 0.33  | 20500 | 3.4148          | 3.4148 | 1.3961 | 0.4016 | 0.2488   |
| 3.542         | 0.34  | 21000 | 3.5980          | 3.5980 | 1.4508 | 0.3695 | 0.2244   |
| 3.5081        | 0.34  | 21500 | 3.6310          | 3.6310 | 1.4488 | 0.3637 | 0.2372   |
| 3.7745        | 0.35  | 22000 | 3.5246          | 3.5246 | 1.4294 | 0.3824 | 0.2378   |
| 3.5048        | 0.36  | 22500 | 3.4395          | 3.4395 | 1.4126 | 0.3973 | 0.241    |
| 3.6374        | 0.37  | 23000 | 3.3863          | 3.3863 | 1.3928 | 0.4066 | 0.247    |
| 3.5231        | 0.38  | 23500 | 3.5991          | 3.5991 | 1.4468 | 0.3693 | 0.2348   |
| 3.5893        | 0.38  | 24000 | 3.2910          | 3.2910 | 1.3692 | 0.4233 | 0.2504   |
| 3.5051        | 0.39  | 24500 | 3.3765          | 3.3765 | 1.3953 | 0.4083 | 0.2394   |
| 3.6082        | 0.4   | 25000 | 3.3060          | 3.3060 | 1.3830 | 0.4207 | 0.2412   |
| 3.4009        | 0.41  | 25500 | 3.4448          | 3.4448 | 1.4095 | 0.3964 | 0.2404   |
| 3.4239        | 0.42  | 26000 | 3.4127          | 3.4127 | 1.4027 | 0.4020 | 0.2412   |
| 3.6036        | 0.42  | 26500 | 3.5339          | 3.5339 | 1.4405 | 0.3808 | 0.2266   |
| 3.4107        | 0.43  | 27000 | 3.3319          | 3.3319 | 1.3776 | 0.4162 | 0.2542   |
| 3.3903        | 0.44  | 27500 | 3.4434          | 3.4434 | 1.4072 | 0.3966 | 0.2486   |
| 3.5583        | 0.45  | 28000 | 3.3119          | 3.3119 | 1.3728 | 0.4197 | 0.2516   |
| 3.4701        | 0.46  | 28500 | 3.3733          | 3.3733 | 1.3910 | 0.4089 | 0.2494   |
| 3.4113        | 0.46  | 29000 | 3.4144          | 3.4144 | 1.4027 | 0.4017 | 0.2414   |
| 3.5731        | 0.47  | 29500 | 3.3822          | 3.3822 | 1.3911 | 0.4073 | 0.2428   |
| 3.5738        | 0.48  | 30000 | 3.4408          | 3.4408 | 1.4120 | 0.3971 | 0.2386   |
| 3.481         | 0.49  | 30500 | 3.3255          | 3.3255 | 1.3794 | 0.4173 | 0.2514   |
| 3.4716        | 0.5   | 31000 | 3.2817          | 3.2817 | 1.3703 | 0.4250 | 0.2492   |
| 3.5487        | 0.5   | 31500 | 3.3388          | 3.3388 | 1.3851 | 0.4149 | 0.2472   |
| 3.2559        | 0.51  | 32000 | 3.3552          | 3.3552 | 1.3847 | 0.4121 | 0.249    |
| 3.5715        | 0.52  | 32500 | 3.2896          | 3.2896 | 1.3692 | 0.4236 | 0.251    |
| 3.4085        | 0.53  | 33000 | 3.2690          | 3.2690 | 1.3685 | 0.4272 | 0.2522   |
| 3.5582        | 0.54  | 33500 | 3.3228          | 3.3228 | 1.3800 | 0.4178 | 0.2462   |
| 3.4105        | 0.54  | 34000 | 3.4462          | 3.4462 | 1.4089 | 0.3961 | 0.2474   |
| 3.5401        | 0.55  | 34500 | 3.3181          | 3.3181 | 1.3751 | 0.4186 | 0.2558   |
| 3.4213        | 0.56  | 35000 | 3.2455          | 3.2455 | 1.3592 | 0.4313 | 0.2548   |
| 3.4644        | 0.57  | 35500 | 3.3900          | 3.3900 | 1.4004 | 0.4060 | 0.2388   |
| 3.4277        | 0.58  | 36000 | 3.2150          | 3.2150 | 1.3506 | 0.4366 | 0.2558   |
| 3.3376        | 0.58  | 36500 | 3.3522          | 3.3522 | 1.3944 | 0.4126 | 0.24     |
| 3.4311        | 0.59  | 37000 | 3.4152          | 3.4152 | 1.3980 | 0.4016 | 0.2498   |
| 3.336         | 0.6   | 37500 | 3.2996          | 3.2996 | 1.3674 | 0.4218 | 0.2594   |
| 3.3557        | 0.61  | 38000 | 3.2040          | 3.2040 | 1.3499 | 0.4386 | 0.2486   |
| 3.3586        | 0.62  | 38500 | 3.2784          | 3.2784 | 1.3632 | 0.4255 | 0.2534   |
| 3.3187        | 0.62  | 39000 | 3.3466          | 3.3466 | 1.3832 | 0.4136 | 0.2468   |
| 3.3899        | 0.63  | 39500 | 3.3209          | 3.3209 | 1.3795 | 0.4181 | 0.25     |
| 3.4483        | 0.64  | 40000 | 3.4685          | 3.4685 | 1.4165 | 0.3922 | 0.2436   |
| 3.3463        | 0.65  | 40500 | 3.3874          | 3.3874 | 1.3961 | 0.4064 | 0.2448   |
| 3.373         | 0.66  | 41000 | 3.2243          | 3.2243 | 1.3518 | 0.4350 | 0.2562   |
| 3.4526        | 0.66  | 41500 | 3.2819          | 3.2819 | 1.3693 | 0.4249 | 0.253    |
| 3.3581        | 0.67  | 42000 | 3.3412          | 3.3412 | 1.3843 | 0.4145 | 0.2456   |
| 3.4551        | 0.68  | 42500 | 3.2484          | 3.2484 | 1.3594 | 0.4308 | 0.2574   |
| 3.4022        | 0.69  | 43000 | 3.2010          | 3.2010 | 1.3468 | 0.4391 | 0.2568   |
| 3.3281        | 0.7   | 43500 | 3.3184          | 3.3184 | 1.3764 | 0.4185 | 0.2476   |
| 3.4044        | 0.7   | 44000 | 3.2361          | 3.2361 | 1.3528 | 0.4329 | 0.2506   |
| 3.3427        | 0.71  | 44500 | 3.2269          | 3.2269 | 1.3557 | 0.4346 | 0.2492   |
| 3.4106        | 0.72  | 45000 | 3.2758          | 3.2758 | 1.3733 | 0.4260 | 0.2434   |
| 3.4406        | 0.73  | 45500 | 3.2235          | 3.2235 | 1.3548 | 0.4352 | 0.2526   |
| 3.491         | 0.74  | 46000 | 3.2842          | 3.2842 | 1.3688 | 0.4245 | 0.2496   |
| 3.4671        | 0.74  | 46500 | 3.1811          | 3.1811 | 1.3464 | 0.4426 | 0.249    |
| 3.5774        | 0.75  | 47000 | 3.2649          | 3.2649 | 1.3608 | 0.4279 | 0.251    |
| 3.4953        | 0.76  | 47500 | 3.2681          | 3.2681 | 1.3616 | 0.4273 | 0.2538   |
| 3.4212        | 0.77  | 48000 | 3.4407          | 3.4407 | 1.4088 | 0.3971 | 0.2424   |
| 3.3285        | 0.78  | 48500 | 3.3279          | 3.3279 | 1.3771 | 0.4169 | 0.2454   |
| 3.361         | 0.78  | 49000 | 3.3717          | 3.3717 | 1.3910 | 0.4092 | 0.243    |
| 3.5419        | 0.79  | 49500 | 3.2851          | 3.2851 | 1.3748 | 0.4244 | 0.2448   |
| 3.3979        | 0.8   | 50000 | 3.3991          | 3.3991 | 1.4039 | 0.4044 | 0.2378   |
| 3.3354        | 0.81  | 50500 | 3.2636          | 3.2636 | 1.3650 | 0.4281 | 0.2456   |
| 3.4488        | 0.82  | 51000 | 3.2604          | 3.2604 | 1.3695 | 0.4287 | 0.243    |
| 3.2583        | 0.82  | 51500 | 3.2759          | 3.2759 | 1.3759 | 0.4260 | 0.2442   |
| 3.3419        | 0.83  | 52000 | 3.2789          | 3.2789 | 1.3728 | 0.4254 | 0.2494   |
| 3.4243        | 0.84  | 52500 | 3.2993          | 3.2993 | 1.3772 | 0.4219 | 0.2486   |
| 3.3154        | 0.85  | 53000 | 3.2350          | 3.2350 | 1.3585 | 0.4331 | 0.2528   |
| 3.3462        | 0.86  | 53500 | 3.2361          | 3.2361 | 1.3594 | 0.4329 | 0.2516   |
| 3.4554        | 0.86  | 54000 | 3.2307          | 3.2307 | 1.3548 | 0.4339 | 0.2528   |
| 3.5053        | 0.87  | 54500 | 3.1970          | 3.1970 | 1.3494 | 0.4398 | 0.2526   |
| 3.2745        | 0.88  | 55000 | 3.2506          | 3.2506 | 1.3614 | 0.4304 | 0.2546   |
| 3.3788        | 0.89  | 55500 | 3.2090          | 3.2090 | 1.3540 | 0.4377 | 0.2516   |
| 3.3216        | 0.9   | 56000 | 3.3347          | 3.3347 | 1.3857 | 0.4157 | 0.2462   |
| 3.2991        | 0.9   | 56500 | 3.1590          | 3.1590 | 1.3397 | 0.4465 | 0.2528   |
| 3.175         | 0.91  | 57000 | 3.2950          | 3.2950 | 1.3734 | 0.4226 | 0.2534   |
| 3.4697        | 0.92  | 57500 | 3.2021          | 3.2021 | 1.3483 | 0.4389 | 0.255    |
| 3.2413        | 0.93  | 58000 | 3.2157          | 3.2157 | 1.3523 | 0.4365 | 0.2518   |
| 3.3949        | 0.94  | 58500 | 3.2709          | 3.2709 | 1.3678 | 0.4268 | 0.2494   |
| 3.3502        | 0.94  | 59000 | 3.2263          | 3.2263 | 1.3558 | 0.4347 | 0.253    |
| 3.3492        | 0.95  | 59500 | 3.2667          | 3.2667 | 1.3659 | 0.4276 | 0.2538   |
| 3.3568        | 0.96  | 60000 | 3.1717          | 3.1717 | 1.3410 | 0.4442 | 0.2542   |
| 3.3886        | 0.97  | 60500 | 3.1800          | 3.1800 | 1.3444 | 0.4428 | 0.2534   |
| 3.2994        | 0.98  | 61000 | 3.2166          | 3.2166 | 1.3539 | 0.4364 | 0.2498   |
| 3.3381        | 0.98  | 61500 | 3.1964          | 3.1964 | 1.3484 | 0.4399 | 0.2534   |
| 3.351         | 0.99  | 62000 | 3.1664          | 3.1664 | 1.3393 | 0.4452 | 0.2538   |
| 3.4063        | 1.0   | 62500 | 3.1764          | 3.1764 | 1.3421 | 0.4434 | 0.2542   |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1
