pytorch roberta token-classification transformers generated_from_trainer license:mit model-index autotrain_compatible
---
license: mit
tags:
- generated_from_trainer
metrics:
- precision
- recall
- accuracy
model-index:
- name: definition-ner
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# definition-ner

This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8437
- Precision: 0.5601
- Recall: 0.6051
- Macro F1: 0.2731
- Micro F1: 0.5817
- Accuracy: 0.8511
- Alias Term F1: 0.5684
- Alias Term Precision: 0.4909
- Alias Term Recall: 0.675
- Definition F1: 0.5130
- Definition Precision: 0.4927
- Definition Recall: 0.5350
- Definition Frag F1: 0.2222
- Definition Frag Precision: 0.1667
- Definition Frag Recall: 0.3333
- Ordered Definition F1: 0.0
- Ordered Definition Precision: 0.0
- Ordered Definition Recall: 0.0
- Ordered Term F1: 0.0
- Ordered Term Precision: 0.0
- Ordered Term Recall: 0.0
- Qualifier F1: 0.0
- Qualifier Precision: 0.0
- Qualifier Recall: 0.0
- Referential Definition F1: 0.3429
- Referential Definition Precision: 0.3158
- Referential Definition Recall: 0.375
- Referential Term F1: 0.2353
- Referential Term Precision: 0.1667
- Referential Term Recall: 0.4
- Secondary Definition F1: 0.1395
- Secondary Definition Precision: 0.12
- Secondary Definition Recall: 0.1667
- Term F1: 0.7101
- Term Precision: 0.7164
- Term Recall: 0.7040

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | Macro F1 | Micro F1 | Accuracy | Alias Term F1 | Alias Term Precision | Alias Term Recall | Definition F1 | Definition Precision | Definition Recall | Definition Frag F1 | Definition Frag Precision | Definition Frag Recall | Ordered Definition F1 | Ordered Definition Precision | Ordered Definition Recall | Ordered Term F1 | Ordered Term Precision | Ordered Term Recall | Qualifier F1 | Qualifier Precision | Qualifier Recall | Referential Definition F1 | Referential Definition Precision | Referential Definition Recall | Referential Term F1 | Referential Term Precision | Referential Term Recall | Secondary Definition F1 | Secondary Definition Precision | Secondary Definition Recall | Term F1 | Term Precision | Term Recall |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:--------:|:--------:|:--------:|:-------------:|:--------------------:|:-----------------:|:-------------:|:--------------------:|:-----------------:|:------------------:|:-------------------------:|:----------------------:|:---------------------:|:----------------------------:|:-------------------------:|:---------------:|:----------------------:|:-------------------:|:------------:|:-------------------:|:----------------:|:-------------------------:|:--------------------------------:|:-----------------------------:|:-------------------:|:--------------------------:|:-----------------------:|:-----------------------:|:------------------------------:|:---------------------------:|:-------:|:--------------:|:-----------:|
| 0.8153        | 1.0   | 756  | 0.4755          | 0.4503    | 0.4510 | 0.1694   | 0.4506   | 0.8450   | 0.25          | 0.3571               | 0.1923            | 0.3563        | 0.3208               | 0.4007            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.5                       | 0.375                            | 0.75                          | 0.0                 | 0.0                        | 0.0                     | 0.0                     | 0.0                            | 0.0                         | 0.5875  | 0.6290         | 0.5511      |
| 0.3215        | 2.0   | 1512 | 0.4521          | 0.4501    | 0.6266 | 0.2020   | 0.5239   | 0.8516   | 0.3248        | 0.2088               | 0.7308            | 0.4761        | 0.4184               | 0.5522            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.5926                    | 0.4211                           | 1.0                           | 0.0                 | 0.0                        | 0.0                     | 0.0                     | 0.0                            | 0.0                         | 0.6262  | 0.5461         | 0.7337      |
| 0.2626        | 3.0   | 2268 | 0.4585          | 0.5123    | 0.6120 | 0.2241   | 0.5577   | 0.8557   | 0.5079        | 0.4324               | 0.6154            | 0.4738        | 0.4093               | 0.5623            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.1429       | 0.1111              | 0.2              | 0.4138                    | 0.2857                           | 0.75                          | 0.0                 | 0.0                        | 0.0                     | 0.0                     | 0.0                            | 0.0                         | 0.7026  | 0.6994         | 0.7059      |
| 0.2078        | 4.0   | 3024 | 0.4552          | 0.5337    | 0.5915 | 0.2097   | 0.5611   | 0.8653   | 0.4857        | 0.3864               | 0.6538            | 0.5085        | 0.4688               | 0.5556            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.4444                    | 0.4                              | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0                     | 0.0                            | 0.0                         | 0.6586  | 0.6431         | 0.6749      |
| 0.1621        | 5.0   | 3780 | 0.4999          | 0.5316    | 0.6032 | 0.2259   | 0.5652   | 0.8537   | 0.5556        | 0.5357               | 0.5769            | 0.5053        | 0.4588               | 0.5623            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.4706                    | 0.4444                           | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0426                  | 0.0333                         | 0.0588                      | 0.6849  | 0.6737         | 0.6966      |
| 0.1291        | 6.0   | 4536 | 0.5596          | 0.5161    | 0.6559 | 0.2294   | 0.5777   | 0.8529   | 0.5152        | 0.425                | 0.6538            | 0.5098        | 0.4365               | 0.6128            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.5                       | 0.375                            | 0.75                          | 0.0                 | 0.0                        | 0.0                     | 0.0625                  | 0.0667                         | 0.0588                      | 0.7066  | 0.6685         | 0.7492      |
| 0.1028        | 7.0   | 5292 | 0.6322          | 0.5481    | 0.6340 | 0.2242   | 0.5879   | 0.8591   | 0.5231        | 0.4359               | 0.6538            | 0.5509        | 0.5146               | 0.5926            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.1429       | 0.1111              | 0.2              | 0.3333                    | 0.25                             | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0                     | 0.0                            | 0.0                         | 0.6922  | 0.6601         | 0.7276      |
| 0.0894        | 8.0   | 6048 | 0.7350          | 0.5692    | 0.6325 | 0.2430   | 0.5992   | 0.8615   | 0.5217        | 0.4186               | 0.6923            | 0.5508        | 0.5071               | 0.6027            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.2222       | 0.25                | 0.2              | 0.3478                    | 0.2667                           | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0870                  | 0.1667                         | 0.0588                      | 0.7003  | 0.6918         | 0.7090      |
| 0.0708        | 9.0   | 6804 | 0.6997          | 0.5572    | 0.6633 | 0.2369   | 0.6056   | 0.8554   | 0.5806        | 0.5                  | 0.6923            | 0.5693        | 0.5150               | 0.6364            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.4706                    | 0.4444                           | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0465                  | 0.0385                         | 0.0588                      | 0.7016  | 0.6621         | 0.7461      |
| 0.0603        | 10.0  | 7560 | 0.7696          | 0.5805    | 0.6545 | 0.2423   | 0.6153   | 0.8632   | 0.6000        | 0.5294               | 0.6923            | 0.5564        | 0.5027               | 0.6229            | 0.0                | 0.0                       | 0.0                    | 0.0                   | 0.0                          | 0.0                       | 0.0             | 0.0                    | 0.0                 | 0.0          | 0.0                 | 0.0              | 0.4706                    | 0.4444                           | 0.5                           | 0.0                 | 0.0                        | 0.0                     | 0.0714                  | 0.0909                         | 0.0588                      | 0.7242  | 0.7092         | 0.7399      |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1
