To produce BioELECTRA, we pretrain ELECTRA on a corpus of over 20 million abstracts from PubMed.

How to use the generator in transformers:

    from transformers import ElectraForMaskedLM, ElectraTokenizerFast
    import torch
    generator = ElectraForMaskedLM.from_pretrained("molly-hayward/bioelectra-base-generator")
    tokenizer = ElectraTokenizerFast.from_pretrained("molly-hayward/bioelectra-base-generator")
{}
transformers pytorch tf electra pretraining endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #electra #pretraining #endpoints_compatible #region-us 

To produce BioELECTRA, we pretrain ELECTRA on a corpus of over 20 million abstracts from PubMed.

How to use the generator in transformers:

    from transformers import ElectraForMaskedLM, ElectraTokenizerFast
    import torch
    generator = ElectraForMaskedLM.from_pretrained("molly-hayward/bioelectra-base-generator")
    tokenizer = ElectraTokenizerFast.from_pretrained("molly-hayward/bioelectra-base-generator")
TAGS
#transformers #pytorch #tf #electra #pretraining #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #tf #electra #pretraining #endpoints_compatible #region-us 
