---
base_model: AIRI-Institute/gena-lm-bert-large-t2t
tags:
- generated_from_trainer
metrics:
- precision
- recall
- accuracy
model-index:
- name: gena-lm-bert-large-t2t_ft_BioS2_1kbpHG19_DHSs_H3K27AC
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gena-lm-bert-large-t2t_ft_BioS2_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of [AIRI-Institute/gena-lm-bert-large-t2t](https://huggingface.co/AIRI-Institute/gena-lm-bert-large-t2t) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3605
- F1 Score: 0.8722
- Precision: 0.8099
- Recall: 0.9449
- Accuracy: 0.8549
- Auc: 0.9416
- Prc: 0.9432

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | F1 Score | Precision | Recall | Accuracy | Auc    | Prc    |
|:-------------:|:------:|:-----:|:---------------:|:--------:|:---------:|:------:|:--------:|:------:|:------:|
| 0.5815        | 0.0840 | 500   | 0.5396          | 0.7416   | 0.8093    | 0.6843 | 0.7500   | 0.8334 | 0.8249 |
| 0.4864        | 0.1680 | 1000  | 0.4740          | 0.8142   | 0.7803    | 0.8513 | 0.7964   | 0.8687 | 0.8579 |
| 0.4701        | 0.2520 | 1500  | 0.5119          | 0.7828   | 0.8415    | 0.7317 | 0.7872   | 0.8688 | 0.8706 |
| 0.4654        | 0.3360 | 2000  | 0.4932          | 0.8274   | 0.7359    | 0.9449 | 0.7934   | 0.8922 | 0.8869 |
| 0.4376        | 0.4200 | 2500  | 0.4289          | 0.8365   | 0.8007    | 0.8756 | 0.8206   | 0.8984 | 0.8962 |
| 0.423         | 0.5039 | 3000  | 0.4881          | 0.8366   | 0.7595    | 0.9311 | 0.8093   | 0.9056 | 0.9034 |
| 0.4226        | 0.5879 | 3500  | 0.3795          | 0.8504   | 0.8110    | 0.8939 | 0.8352   | 0.9131 | 0.9133 |
| 0.4035        | 0.6719 | 4000  | 0.3801          | 0.8491   | 0.7886    | 0.9196 | 0.8287   | 0.9186 | 0.9217 |
| 0.4041        | 0.7559 | 4500  | 0.3687          | 0.8553   | 0.8172    | 0.8971 | 0.8409   | 0.9226 | 0.9244 |
| 0.3948        | 0.8399 | 5000  | 0.4052          | 0.8421   | 0.7465    | 0.9657 | 0.8102   | 0.9290 | 0.9314 |
| 0.3914        | 0.9239 | 5500  | 0.4325          | 0.8483   | 0.7639    | 0.9538 | 0.8213   | 0.9294 | 0.9319 |
| 0.3801        | 1.0079 | 6000  | 0.3344          | 0.8677   | 0.8623    | 0.8731 | 0.8604   | 0.9339 | 0.9368 |
| 0.3334        | 1.0919 | 6500  | 0.3710          | 0.8695   | 0.8301    | 0.9128 | 0.8564   | 0.9337 | 0.9368 |
| 0.3567        | 1.1759 | 7000  | 0.3799          | 0.8672   | 0.8740    | 0.8606 | 0.8619   | 0.9372 | 0.9398 |
| 0.3528        | 1.2599 | 7500  | 0.3344          | 0.8698   | 0.8609    | 0.8788 | 0.8621   | 0.9378 | 0.9404 |
| 0.3542        | 1.3439 | 8000  | 0.3936          | 0.8769   | 0.8249    | 0.9359 | 0.8623   | 0.9393 | 0.9405 |
| 0.3497        | 1.4279 | 8500  | 0.3461          | 0.8730   | 0.8586    | 0.8878 | 0.8646   | 0.9393 | 0.9403 |
| 0.3379        | 1.5118 | 9000  | 0.4219          | 0.8683   | 0.7987    | 0.9513 | 0.8488   | 0.9408 | 0.9417 |
| 0.347         | 1.5958 | 9500  | 0.3649          | 0.8746   | 0.8491    | 0.9016 | 0.8644   | 0.9391 | 0.9409 |
| 0.3513        | 1.6798 | 10000 | 0.3315          | 0.8655   | 0.8944    | 0.8385 | 0.8634   | 0.9414 | 0.9438 |
| 0.3259        | 1.7638 | 10500 | 0.3605          | 0.8722   | 0.8099    | 0.9449 | 0.8549   | 0.9416 | 0.9432 |


### Framework versions

- Transformers 4.42.3
- Pytorch 2.3.0+cu121
- Datasets 2.18.0
- Tokenizers 0.19.0