This is a model checkpoint for "Should You Mask 15% in Masked Language Modeling" (code). We use pre layer norm, which is not supported by HuggingFace. To use our model, go to our github repo, download our code, and import the RoBERTa class from huggingface/modeling_roberta_prelayernorm.py. For example,

from huggingface.modeling_roberta_prelayernorm import RobertaForMaskedLM, RobertaForSequenceClassification

Downloads last month: 6

Inference Providers NEW

Fill-Mask

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model authors have turned it off explicitly.