
[`bioformer-8L`](https://huggingface.co/bioformers/bioformer-8L) pretrained on 164,179 COVID-19 abstracts (from [LitCovid website](https://www.ncbi.nlm.nih.gov/research/coronavirus/)) for 100 epochs. 

In our evaluation, this pretraining process leads to improved performance on the multi-label COVID-19 topic classification task (BioCreative VII track 5). 
{"language": ["en"], "license": "apache-2.0"}
transformers pytorch bert fill-mask en license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #en #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 


'bioformer-8L' pretrained on 164,179 COVID-19 abstracts (from LitCovid website) for 100 epochs. 

In our evaluation, this pretraining process leads to improved performance on the multi-label COVID-19 topic classification task (BioCreative VII track 5). 
TAGS
#transformers #pytorch #bert #fill-mask #en #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #bert #fill-mask #en #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
