---
license: apache-2.0
base_model: google-bert/bert-base-uncased
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: 20240327184156_red_mikolov
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 20240327184156_red_mikolov

This model is a fine-tuned version of [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0292
- Precision: 0.9583
- Recall: 0.9476
- F1: 0.9529
- Accuracy: 0.9893

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 32
- eval_batch_size: 32
- seed: 69
- gradient_accumulation_steps: 8
- total_train_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 350
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0569        | 0.09  | 300   | 0.0439          | 0.9281    | 0.9188 | 0.9234 | 0.9828   |
| 0.0602        | 0.18  | 600   | 0.0475          | 0.9297    | 0.9080 | 0.9187 | 0.9817   |
| 0.0586        | 0.27  | 900   | 0.0455          | 0.9277    | 0.9117 | 0.9196 | 0.9820   |
| 0.0566        | 0.36  | 1200  | 0.0454          | 0.9269    | 0.9116 | 0.9192 | 0.9821   |
| 0.056         | 0.44  | 1500  | 0.0460          | 0.9362    | 0.9050 | 0.9204 | 0.9824   |
| 0.0549        | 0.53  | 1800  | 0.0430          | 0.9277    | 0.9244 | 0.9260 | 0.9831   |
| 0.0526        | 0.62  | 2100  | 0.0404          | 0.9326    | 0.9218 | 0.9272 | 0.9837   |
| 0.0523        | 0.71  | 2400  | 0.0413          | 0.9313    | 0.9243 | 0.9278 | 0.9836   |
| 0.0524        | 0.8   | 2700  | 0.0402          | 0.9410    | 0.9136 | 0.9271 | 0.9840   |
| 0.0517        | 0.89  | 3000  | 0.0413          | 0.9354    | 0.9198 | 0.9275 | 0.9837   |
| 0.0512        | 0.98  | 3300  | 0.0411          | 0.9360    | 0.9148 | 0.9253 | 0.9836   |
| 0.0442        | 1.07  | 3600  | 0.0406          | 0.9297    | 0.9285 | 0.9291 | 0.9840   |
| 0.0457        | 1.15  | 3900  | 0.0421          | 0.9412    | 0.9161 | 0.9285 | 0.9840   |
| 0.0452        | 1.24  | 4200  | 0.0400          | 0.9343    | 0.9281 | 0.9312 | 0.9847   |
| 0.0444        | 1.33  | 4500  | 0.0381          | 0.9341    | 0.9329 | 0.9335 | 0.9848   |
| 0.0434        | 1.42  | 4800  | 0.0376          | 0.9409    | 0.9268 | 0.9338 | 0.9850   |
| 0.0437        | 1.51  | 5100  | 0.0385          | 0.9407    | 0.9224 | 0.9315 | 0.9847   |
| 0.0424        | 1.6   | 5400  | 0.0364          | 0.9437    | 0.9268 | 0.9352 | 0.9855   |
| 0.042         | 1.69  | 5700  | 0.0370          | 0.9445    | 0.9247 | 0.9345 | 0.9853   |
| 0.0422        | 1.78  | 6000  | 0.0361          | 0.9408    | 0.9320 | 0.9364 | 0.9854   |
| 0.0413        | 1.86  | 6300  | 0.0354          | 0.9426    | 0.9303 | 0.9364 | 0.9857   |
| 0.0406        | 1.95  | 6600  | 0.0353          | 0.9408    | 0.9326 | 0.9367 | 0.9860   |
| 0.0336        | 2.04  | 6900  | 0.0353          | 0.9438    | 0.9342 | 0.9390 | 0.9862   |
| 0.0338        | 2.13  | 7200  | 0.0362          | 0.9500    | 0.9227 | 0.9362 | 0.9860   |
| 0.0341        | 2.22  | 7500  | 0.0356          | 0.9428    | 0.9325 | 0.9376 | 0.9861   |
| 0.0333        | 2.31  | 7800  | 0.0348          | 0.9423    | 0.9350 | 0.9386 | 0.9863   |
| 0.0344        | 2.4   | 8100  | 0.0337          | 0.9454    | 0.9368 | 0.9411 | 0.9869   |
| 0.0338        | 2.49  | 8400  | 0.0336          | 0.9486    | 0.9360 | 0.9422 | 0.9869   |
| 0.0334        | 2.57  | 8700  | 0.0336          | 0.9482    | 0.9332 | 0.9407 | 0.9866   |
| 0.0325        | 2.66  | 9000  | 0.0333          | 0.9491    | 0.9336 | 0.9413 | 0.9868   |
| 0.0323        | 2.75  | 9300  | 0.0320          | 0.9467    | 0.9382 | 0.9424 | 0.9873   |
| 0.0318        | 2.84  | 9600  | 0.0329          | 0.9531    | 0.9267 | 0.9397 | 0.9867   |
| 0.0316        | 2.93  | 9900  | 0.0314          | 0.9497    | 0.9372 | 0.9434 | 0.9874   |
| 0.0246        | 3.02  | 10200 | 0.0336          | 0.9510    | 0.9374 | 0.9441 | 0.9874   |
| 0.0246        | 3.11  | 10500 | 0.0313          | 0.9513    | 0.9435 | 0.9474 | 0.9880   |
| 0.0242        | 3.2   | 10800 | 0.0329          | 0.9500    | 0.9376 | 0.9437 | 0.9876   |
| 0.0248        | 3.29  | 11100 | 0.0313          | 0.9544    | 0.9364 | 0.9453 | 0.9881   |
| 0.0244        | 3.37  | 11400 | 0.0318          | 0.9509    | 0.9429 | 0.9469 | 0.9879   |
| 0.0244        | 3.46  | 11700 | 0.0302          | 0.9546    | 0.9417 | 0.9481 | 0.9882   |
| 0.0245        | 3.55  | 12000 | 0.0308          | 0.9504    | 0.9384 | 0.9444 | 0.9879   |
| 0.0237        | 3.64  | 12300 | 0.0304          | 0.9510    | 0.9401 | 0.9455 | 0.9880   |
| 0.0236        | 3.73  | 12600 | 0.0301          | 0.9572    | 0.9367 | 0.9468 | 0.9881   |
| 0.0232        | 3.82  | 12900 | 0.0299          | 0.9560    | 0.9417 | 0.9488 | 0.9884   |
| 0.0231        | 3.91  | 13200 | 0.0288          | 0.9555    | 0.9446 | 0.9500 | 0.9886   |
| 0.0228        | 4.0   | 13500 | 0.0287          | 0.9553    | 0.9450 | 0.9501 | 0.9886   |
| 0.0169        | 4.08  | 13800 | 0.0313          | 0.9563    | 0.9426 | 0.9494 | 0.9886   |
| 0.0169        | 4.17  | 14100 | 0.0311          | 0.9564    | 0.9434 | 0.9499 | 0.9887   |
| 0.0167        | 4.26  | 14400 | 0.0305          | 0.9562    | 0.9478 | 0.9520 | 0.9889   |
| 0.0166        | 4.35  | 14700 | 0.0304          | 0.9549    | 0.9478 | 0.9513 | 0.9890   |
| 0.0165        | 4.44  | 15000 | 0.0296          | 0.9579    | 0.9453 | 0.9516 | 0.9890   |
| 0.0162        | 4.53  | 15300 | 0.0295          | 0.9562    | 0.9492 | 0.9527 | 0.9892   |
| 0.0158        | 4.62  | 15600 | 0.0291          | 0.9563    | 0.9483 | 0.9523 | 0.9892   |
| 0.0157        | 4.71  | 15900 | 0.0288          | 0.9575    | 0.9505 | 0.9540 | 0.9894   |
| 0.0153        | 4.79  | 16200 | 0.0293          | 0.9580    | 0.9472 | 0.9526 | 0.9892   |
| 0.0152        | 4.88  | 16500 | 0.0292          | 0.9581    | 0.9476 | 0.9528 | 0.9893   |
| 0.0152        | 4.97  | 16800 | 0.0292          | 0.9583    | 0.9476 | 0.9529 | 0.9893   |


### Framework versions

- Transformers 4.39.1
- Pytorch 2.2.0a0+6a974be
- Datasets 2.18.0
- Tokenizers 0.15.2