---
library_name: transformers
license: apache-2.0
base_model: dslim/distilbert-NER
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: distilbert-classn-LinearAlg-finetuned-pred-span-width-2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-classn-LinearAlg-finetuned-pred-span-width-2

This model is a fine-tuned version of [dslim/distilbert-NER](https://huggingface.co/dslim/distilbert-NER) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6166
- Accuracy: 0.8333
- F1: 0.8321
- Precision: 0.8444
- Recall: 0.8333

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step | Validation Loss | Accuracy | F1     | Precision | Recall |
|:-------------:|:-------:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
| 5.0405        | 0.6849  | 50   | 2.4804          | 0.0556   | 0.0339 | 0.0253    | 0.0556 |
| 4.9397        | 1.3699  | 100  | 2.4514          | 0.0714   | 0.0438 | 0.0322    | 0.0714 |
| 4.8593        | 2.0548  | 150  | 2.4005          | 0.0794   | 0.0627 | 0.0607    | 0.0794 |
| 4.7519        | 2.7397  | 200  | 2.3404          | 0.1905   | 0.1699 | 0.1849    | 0.1905 |
| 4.7117        | 3.4247  | 250  | 2.2563          | 0.2698   | 0.2785 | 0.3828    | 0.2698 |
| 4.4979        | 4.1096  | 300  | 2.1144          | 0.3810   | 0.3625 | 0.3976    | 0.3810 |
| 4.1155        | 4.7945  | 350  | 1.9127          | 0.5635   | 0.5591 | 0.6445    | 0.5635 |
| 3.5796        | 5.4795  | 400  | 1.6802          | 0.6032   | 0.6019 | 0.7030    | 0.6032 |
| 3.0998        | 6.1644  | 450  | 1.4151          | 0.6984   | 0.6879 | 0.7921    | 0.6984 |
| 2.5176        | 6.8493  | 500  | 1.1687          | 0.7698   | 0.7665 | 0.7902    | 0.7698 |
| 1.9411        | 7.5342  | 550  | 0.9922          | 0.7619   | 0.7633 | 0.8083    | 0.7619 |
| 1.4025        | 8.2192  | 600  | 0.8374          | 0.8095   | 0.8088 | 0.8457    | 0.8095 |
| 1.0761        | 8.9041  | 650  | 0.7305          | 0.8175   | 0.8124 | 0.8412    | 0.8175 |
| 0.8084        | 9.5890  | 700  | 0.6920          | 0.8254   | 0.8202 | 0.8502    | 0.8254 |
| 0.5516        | 10.2740 | 750  | 0.6456          | 0.8333   | 0.8328 | 0.8705    | 0.8333 |
| 0.4201        | 10.9589 | 800  | 0.6497          | 0.8175   | 0.8102 | 0.8566    | 0.8175 |
| 0.2738        | 11.6438 | 850  | 0.5939          | 0.8333   | 0.8337 | 0.8524    | 0.8333 |
| 0.235         | 12.3288 | 900  | 0.6067          | 0.8413   | 0.8397 | 0.8641    | 0.8413 |
| 0.1387        | 13.0137 | 950  | 0.5975          | 0.8333   | 0.8306 | 0.8496    | 0.8333 |
| 0.1154        | 13.6986 | 1000 | 0.5704          | 0.8413   | 0.8389 | 0.8515    | 0.8413 |
| 0.0715        | 14.3836 | 1050 | 0.5859          | 0.8413   | 0.8397 | 0.8536    | 0.8413 |
| 0.0741        | 15.0685 | 1100 | 0.5732          | 0.8413   | 0.8393 | 0.8510    | 0.8413 |
| 0.0545        | 15.7534 | 1150 | 0.6005          | 0.8333   | 0.8310 | 0.8512    | 0.8333 |
| 0.0354        | 16.4384 | 1200 | 0.6069          | 0.8413   | 0.8398 | 0.8564    | 0.8413 |
| 0.0435        | 17.1233 | 1250 | 0.6056          | 0.8413   | 0.8389 | 0.8515    | 0.8413 |
| 0.0305        | 17.8082 | 1300 | 0.6066          | 0.8413   | 0.8393 | 0.8558    | 0.8413 |
| 0.02          | 18.4932 | 1350 | 0.6091          | 0.8333   | 0.8315 | 0.8427    | 0.8333 |
| 0.0271        | 19.1781 | 1400 | 0.6121          | 0.8333   | 0.8315 | 0.8427    | 0.8333 |
| 0.0211        | 19.8630 | 1450 | 0.6166          | 0.8333   | 0.8321 | 0.8444    | 0.8333 |


### Framework versions

- Transformers 4.48.3
- Pytorch 2.5.1+cu124
- Datasets 3.3.1
- Tokenizers 0.21.0