
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# NLP-CIC-WFU_Clinical_Cases_NER_Sents_tokenized_mBERT_cased_fine_tuned

This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on the LivingNER shared task 2022 dataset. It is available at: https://temu.bsc.es/livingner/category/data/

It achieves the following results on the evaluation set:
- Loss: 0.0546
- Precision: 0.8574
- Recall: 0.7366
- F1: 0.7924
- Accuracy: 0.9893

## Model description

For a complete description of our system, please go to: https://ceur-ws.org/Vol-3202/livingner-paper13.pdf


## Training and evaluation data

Dataset provided by LivingNER shared task, it is available at: https://temu.bsc.es/livingner/category/data/

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 7

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0505        | 1.0   | 2568  | 0.0434          | 0.9399    | 0.6781 | 0.7878 | 0.9886   |
| 0.0393        | 2.0   | 5136  | 0.0450          | 0.9384    | 0.6947 | 0.7984 | 0.9892   |
| 0.0306        | 3.0   | 7704  | 0.0451          | 0.9497    | 0.6951 | 0.8027 | 0.9897   |
| 0.0266        | 4.0   | 10272 | 0.0422          | 0.9646    | 0.6904 | 0.8048 | 0.9900   |
| 0.0208        | 5.0   | 12840 | 0.0494          | 0.9576    | 0.6969 | 0.8067 | 0.9902   |
| 0.0141        | 6.0   | 15408 | 0.0506          | 0.8407    | 0.7352 | 0.7844 | 0.9890   |
| 0.0093        | 7.0   | 17976 | 0.0546          | 0.8574    | 0.7366 | 0.7924 | 0.9893   |

### How to cite this work:

Tamayo, A., Burgos, D., & Gelbukh, A. (2022). ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules. In CEUR Workshop Proceedings (Vol. 3202). CEUR-WS.

@inproceedings{tamayo2022partner,
  title={ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules},
  author={Tamayo, Antonio and Burgos, Diego and Gelbukh, Alexander}
}

### Framework versions

- Transformers 4.19.2
- Pytorch 1.11.0+cu113
- Datasets 2.2.2
- Tokenizers 0.12.1

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "metrics": ["precision", "recall", "f1", "accuracy"], "widget": [{"text": "Se hospitaliz\u00f3 un hombre de 42 a\u00f1os, al que se le hab\u00eda diagnosticado recientemente un carcinoma renal sarcomatoide de c\u00e9lulas claras metast\u00e1sico, con fiebre, manejo del dolor por met\u00e1stasis \u00f3seas sintom\u00e1ticas y para decisiones de tratamiento sist\u00e9mico de primera l\u00ednea. El paciente no ten\u00eda otros antecedentes. Inicialmente present\u00f3 fiebre de 39,0 \u00b0C el 12 de marzo de 2020, para la cual recibi\u00f3 ceftriaxona fuera de nuestro centro. El d\u00eda 6, present\u00f3 tos leve y fiebre (38,3\u00b0C), lo que llev\u00f3 a realizar una prueba de PCR en tiempo real para SARS-CoV-2; el resultado fue positivo. El paciente fue ingresado en la sala de COVID-19 de nuestro hospital y se monitoriz\u00f3 estrechamente. La TAC tor\u00e1cica mostr\u00f3 opacidades de vidrio esmerilado bilaterales parcheadas, asociadas al COVID-19 (figura 1). El D7 se le empez\u00f3 a administrar terapia antiv\u00edrica con lopinavir y ritonavir (400mg/100mg por v\u00eda oral), que se mantuvo durante 5 d\u00edas, seg\u00fan las directrices locales. El d\u00eda 8, una disnea s\u00fabita y una ca\u00edda de la saturaci\u00f3n obligaron a aumentar el ox\u00edgeno a 6 l/min, sin necesidad de ventilaci\u00f3n mec\u00e1nica. Se le administraron dos dosis de tocilizumab, con 8 mg/kg i.v. en cada dosis, separadas 8 horas, con buena tolerancia. Despu\u00e9s mostr\u00f3 una mejora cl\u00ednica, pasando a afebril r\u00e1pidamente y con un consumo de ox\u00edgeno decreciente, que fue retirado por completo el d\u00eda 12. Una TAC tor\u00e1cica del d\u00eda 12 confirm\u00f3 la mejora mostrando regresi\u00f3n parcial de los infiltrados pulmonares y de las opacidades de vidrio esmerilado. La prote\u00edna C-reactiva, un marcador indirecto de liberaci\u00f3n de citocinas, disminuy\u00f3 de 225 mg/L a 33 mg/L en 4 d\u00edas (figura 1). Tras la administraci\u00f3n de tocilizumab no se observaron cambios relevantes en las subpoblaciones linfoc\u00edticas circulantes y el porcentaje de CD4 + CD25 + linfocitos era alto, antes y despu\u00e9s del tocilizumab. Finalmente, el paciente se recuper\u00f3 totalmente de los s\u00edntomas de la COVID-19."}], "model-index": [{"name": "NLP-CIC-WFU_Clinical_Cases_NER_Sents_tokenized_mBERT_cased_fine_tuned", "results": []}]}
transformers pytorch tensorboard bert token-classification generated_from_trainer license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #bert #token-classification #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

NLP-CIC-WFU\_Clinical\_Cases\_NER\_Sents\_tokenized\_mBERT\_cased\_fine\_tuned
==============================================================================


This model is a fine-tuned version of bert-base-multilingual-cased on the LivingNER shared task 2022 dataset. It is available at: URL


It achieves the following results on the evaluation set:


* Loss: 0.0546
* Precision: 0.8574
* Recall: 0.7366
* F1: 0.7924
* Accuracy: 0.9893


Model description
-----------------


For a complete description of our system, please go to: URL


Training and evaluation data
----------------------------


Dataset provided by LivingNER shared task, it is available at: URL


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7


### Training results



### How to cite this work:


Tamayo, A., Burgos, D., & Gelbukh, A. (2022). ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules. In CEUR Workshop Proceedings (Vol. 3202). CEUR-WS.


@inproceedings{tamayo2022partner,
title={ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules},
author={Tamayo, Antonio and Burgos, Diego and Gelbukh, Alexander}
}


### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0+cu113
* Datasets 2.2.2
* Tokenizers 0.12.1



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7 ### Training results ### How to cite this work:


Tamayo, A., Burgos, D., & Gelbukh, A. (2022). ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules. In CEUR Workshop Proceedings (Vol. 3202). CEUR-WS.


@inproceedings{tamayo2022partner,
title={ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules},
author={Tamayo, Antonio and Burgos, Diego and Gelbukh, Alexander}
} ### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0+cu113
* Datasets 2.2.2
* Tokenizers 0.12.1
TAGS
#transformers #pytorch #tensorboard #bert #token-classification #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7 ### Training results ### How to cite this work:


Tamayo, A., Burgos, D., & Gelbukh, A. (2022). ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules. In CEUR Workshop Proceedings (Vol. 3202). CEUR-WS.


@inproceedings{tamayo2022partner,
title={ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules},
author={Tamayo, Antonio and Burgos, Diego and Gelbukh, Alexander}
} ### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0+cu113
* Datasets 2.2.2
* Tokenizers 0.12.1
passage: TAGS
#transformers #pytorch #tensorboard #bert #token-classification #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 7### Training results### How to cite this work:


Tamayo, A., Burgos, D., & Gelbukh, A. (2022). ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules. In CEUR Workshop Proceedings (Vol. 3202). CEUR-WS.


@inproceedings{tamayo2022partner,
title={ParTNER: Paragraph Tuning for Named Entity Recognition on Clinical Cases in Spanish using mBERT+ Rules},
author={Tamayo, Antonio and Burgos, Diego and Gelbukh, Alexander}
}### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0+cu113
* Datasets 2.2.2
* Tokenizers 0.12.1