How to implement or avoid masking for transformer?
If you're using a library such as Trax which contains great submodules for various Transformers (Skipping, BERT, Vanilla and Reformer) you can use the inbuilt trax.data.inputs.add_loss_weights() function and provide a value for the id_to_mask parameter.
Example Usage:
train_generator = trax.data.inputs.add_loss_weights(
data_generator(batch_size, x_train, y_train,vocab['&lt;PAD&gt;'], True),
id_to_mask=vocab['&lt;PAD&gt;'])

Here are some resources for building Transformers in Trax:

Named Entity Recognition with Reformer (Masking Implemented) and the associated blog post (contains an in-depth explanation of NER and Transformers)

Deep N-Gram Models (without masking but I recommend you go through these first to get a hang of the library)