Masked Language Modeling on Domain-specific Data
First I suggest reading the transformers paper. Couple of quick notes is that this model consists of an encoder and a decoder, and the original task the paper is trained on is machine translation. Datasets (benchmarks) they used to train and evaluate this model from scratch were WMT 2014 Engligh-to-German, WMT 2014 English-to-French (section 5.1 of the paper). The conclusion is that you cannot train transformers from scratch unless you have your sentence pairs in 2 languages.
MLM on the other hand is something that BERT used for training. So if you want to go in this direction, you can use the pre-trained BERT and fine-tune it with Masked language model head on top of your BERT model using your own dataset. You need to stick to the tokenizer that BERT was originally trained on but at least you are taking advantage of some general context that was learned during training BERT from scratch. If you want to train BERT either from scratch or fine-tune with MLM head you can follow this tutorial from hugging face