
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# roberta-es-clinical-trials-medic-attr-ner

This named entity recognition model detects medication-related information:
- Contraindication: e.g. *contraindicación a **aspirina***
- Dose, strength or concentration: e.g. *14 mg*, *100.000 UI*
- Form: e.g. *tabletas*, *comprimidos*
- Route: e.g. *vía oral*, *i.v.*

The model achieves the following results on the test set (when trained with the training and development set; results are averaged over 5 evaluation rounds):
- Precision: 0.863 (±0.011)
- Recall: 0.878 (±0.008)
- F1: 0.871 (±0.001)
- Accuracy: 0.997 (±0.001)


## Model description

This model adapts the pre-trained model [bsc-bio-ehr-es](https://huggingface.co/PlanTL-GOB-ES/bsc-bio-ehr-es), presented in [Pio Carriño et al. (2022)](https://aclanthology.org/2022.bionlp-1.19/). 
It is fine-tuned to conduct temporal named entity recognition on Spanish texts about clinical trials. 
The model is fine-tuned on the [CT-EBM-ES corpus (Campillos-Llanos et al. 2021)](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-021-01395-z).

## Intended uses & limitations

**Disclosure**: *This model is under development and needs to be improved. It should not be used for medical decision making without human assistance and supervision*

This model is intended for a generalist purpose, and may have bias and/or any other undesirable distortions.

Third parties who deploy or provide systems and/or services using any of these models (or using systems based on these models) should note that it is their responsibility to mitigate the risks arising from their use. Third parties, in any event, need to comply with applicable regulations, including regulations concerning the use of artificial intelligence.

The owner or creator of the models will in no event be liable for any results arising from the use made by third parties of these models.

**Descargo de responsabilidad**: *Esta herramienta se encuentra en desarrollo y no debe ser empleada para la toma de decisiones médicas*

La finalidad de este modelo es generalista, y se advierte que puede tener sesgos y/u otro tipo de distorsiones indeseables.

Terceras partes que desplieguen o proporcionen sistemas y/o servicios usando alguno de estos modelos (o utilizando sistemas basados en estos modelos) han tener presente que es su responsabilidad abordar y minimizar los riesgos derivados de su uso. Las terceras partes, en cualquier circunstancia, deben cumplir con la normativa aplicable, incluyendo la normativa que concierne al uso de la inteligencia artificial.

El propietario o creador de los modelos de ningún modo será responsable de los resultados derivados del uso que las terceras partes hagan de estos modelos.


## Training and evaluation data

The data used for fine-tuning are the [Clinical Trials for Evidence-Based-Medicine in Spanish corpus](http://www.lllf.uam.es/ESP/nlpdata/wp2/).
It is a collection of 1200 texts about clinical trials studies and clinical trials announcements:
- 500 abstracts from journals published under a Creative Commons license, e.g. available in PubMed or the Scientific Electronic Library Online (SciELO)
- 700 clinical trials announcements published in the European Clinical Trials Register and Repositorio Español de Estudios Clínicos

If you use the CT-EBM-ES resource, please, cite as follows:

```
@article{campillosetal-midm2021,
        title = {A clinical trials corpus annotated with UMLS© entities to enhance the access to Evidence-Based Medicine},
        author = {Campillos-Llanos, Leonardo and Valverde-Mateos, Ana and Capllonch-Carri{\'o}n, Adri{\'a}n and Moreno-Sandoval, Antonio},
        journal = {BMC Medical Informatics and Decision Making},
        volume={21},
        number={1},
        pages={1--19},
        year={2021},
        publisher={BioMed Central}
}
```



## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: average 12 epochs (±3.1); trained with early stopping if no improvement after 5 epochs (early stopping patience: 5) 


### Training results (test set; average and standard deviation of 5 rounds with different seeds)

|   Precision    |     Recall     |       F1       |    Accuracy    |
|:--------------:|:--------------:|:--------------:|:--------------:|
| 0.863 (±0.011) | 0.878 (±0.008) | 0.871 (±0.001) | 0.997 (±0.001) |


**Results per class (test set; average and standard deviation of 5 rounds with different seeds)**  

|      Class      |   Precision    |     Recall     |       F1       |  Support  |
|:---------------:|:--------------:|:--------------:|:--------------:|:---------:|
| Contraindicated | 0.752 (±0.089) | 0.847 (±0.077) | 0.791 (±0.041) |     76    |
| Dose            | 0.830 (±0.032) | 0.838 (±0.035) | 0.833 (±0.001) |    320    |
| Form            | 0.971 (±0.029) | 0.889 (±0.024) | 0.928 (±0.021) |     74    |
| Route           | 0.934 (±0.012) | 0.916 (±0.024) | 0.925 (±0.012) |    270    |


### Framework versions

- Transformers 4.17.0
- Pytorch 1.10.2+cu113
- Datasets 1.18.4
- Tokenizers 0.11.6

{"license": "cc-by-nc-4.0", "tags": ["generated_from_trainer"], "metrics": ["precision", "recall", "f1", "accuracy"], "widget": [{"text": "Azitromicina en suspensi\u00f3n oral, 10 mg/kg una vez al d\u00eda durante siete d\u00edas"}, {"text": "A un grupo se le administr\u00f3 Ciprofloxacino 200 mg bid EV y al otro Cefazolina 1 g tid IV"}, {"text": "Administraci\u00f3n de una soluci\u00f3n de mantenimiento intravenosa isot\u00f3nica (NaCl al 0,9% en dextrosa al 5%)"}, {"text": "Se excluyen pacientes con contraindicaci\u00f3n a aspirina o clopidogrel"}], "model-index": [{"name": "roberta-es-clinical-trials-medic-attr-ner", "results": []}]}
transformers pytorch roberta token-classification generated_from_trainer license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #roberta #token-classification #generated_from_trainer #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 

roberta-es-clinical-trials-medic-attr-ner
=========================================


This named entity recognition model detects medication-related information:


* Contraindication: e.g. *contraindicación a aspirina*
* Dose, strength or concentration: e.g. *14 mg*, *100.000 UI*
* Form: e.g. *tabletas*, *comprimidos*
* Route: e.g. *vía oral*, *i.v.*


The model achieves the following results on the test set (when trained with the training and development set; results are averaged over 5 evaluation rounds):


* Precision: 0.863 (±0.011)
* Recall: 0.878 (±0.008)
* F1: 0.871 (±0.001)
* Accuracy: 0.997 (±0.001)


Model description
-----------------


This model adapts the pre-trained model bsc-bio-ehr-es, presented in Pio Carriño et al. (2022).
It is fine-tuned to conduct temporal named entity recognition on Spanish texts about clinical trials.
The model is fine-tuned on the CT-EBM-ES corpus (Campillos-Llanos et al. 2021).


Intended uses & limitations
---------------------------


Disclosure: *This model is under development and needs to be improved. It should not be used for medical decision making without human assistance and supervision*


This model is intended for a generalist purpose, and may have bias and/or any other undesirable distortions.


Third parties who deploy or provide systems and/or services using any of these models (or using systems based on these models) should note that it is their responsibility to mitigate the risks arising from their use. Third parties, in any event, need to comply with applicable regulations, including regulations concerning the use of artificial intelligence.


The owner or creator of the models will in no event be liable for any results arising from the use made by third parties of these models.


Descargo de responsabilidad: *Esta herramienta se encuentra en desarrollo y no debe ser empleada para la toma de decisiones médicas*


La finalidad de este modelo es generalista, y se advierte que puede tener sesgos y/u otro tipo de distorsiones indeseables.


Terceras partes que desplieguen o proporcionen sistemas y/o servicios usando alguno de estos modelos (o utilizando sistemas basados en estos modelos) han tener presente que es su responsabilidad abordar y minimizar los riesgos derivados de su uso. Las terceras partes, en cualquier circunstancia, deben cumplir con la normativa aplicable, incluyendo la normativa que concierne al uso de la inteligencia artificial.


El propietario o creador de los modelos de ningún modo será responsable de los resultados derivados del uso que las terceras partes hagan de estos modelos.


Training and evaluation data
----------------------------


The data used for fine-tuning are the Clinical Trials for Evidence-Based-Medicine in Spanish corpus.
It is a collection of 1200 texts about clinical trials studies and clinical trials announcements:


* 500 abstracts from journals published under a Creative Commons license, e.g. available in PubMed or the Scientific Electronic Library Online (SciELO)
* 700 clinical trials announcements published in the European Clinical Trials Register and Repositorio Español de Estudios Clínicos


If you use the CT-EBM-ES resource, please, cite as follows:


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: average 12 epochs (±3.1); trained with early stopping if no improvement after 5 epochs (early stopping patience: 5)


### Training results (test set; average and standard deviation of 5 rounds with different seeds)



Results per class (test set; average and standard deviation of 5 rounds with different seeds)



### Framework versions


* Transformers 4.17.0
* Pytorch 1.10.2+cu113
* Datasets 1.18.4
* Tokenizers 0.11.6



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: average 12 epochs (±3.1); trained with early stopping if no improvement after 5 epochs (early stopping patience: 5) ### Training results (test set; average and standard deviation of 5 rounds with different seeds)



Results per class (test set; average and standard deviation of 5 rounds with different seeds) ### Framework versions


* Transformers 4.17.0
* Pytorch 1.10.2+cu113
* Datasets 1.18.4
* Tokenizers 0.11.6
TAGS
#transformers #pytorch #roberta #token-classification #generated_from_trainer #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: average 12 epochs (±3.1); trained with early stopping if no improvement after 5 epochs (early stopping patience: 5) ### Training results (test set; average and standard deviation of 5 rounds with different seeds)



Results per class (test set; average and standard deviation of 5 rounds with different seeds) ### Framework versions


* Transformers 4.17.0
* Pytorch 1.10.2+cu113
* Datasets 1.18.4
* Tokenizers 0.11.6
passage: TAGS
#transformers #pytorch #roberta #token-classification #generated_from_trainer #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: average 12 epochs (±3.1); trained with early stopping if no improvement after 5 epochs (early stopping patience: 5)### Training results (test set; average and standard deviation of 5 rounds with different seeds)



Results per class (test set; average and standard deviation of 5 rounds with different seeds)### Framework versions


* Transformers 4.17.0
* Pytorch 1.10.2+cu113
* Datasets 1.18.4
* Tokenizers 0.11.6