How are we making prediction for masked tokens alone in BERT?
Is it that the loss is calculated for masked token alone while
ignoring predictions from rest of the tokens.

The quick answer "YES"
Suppose given the following sentence:
"The red apple is my favourite fruit."
I can mask the above sentence as:
"The red apple is my favourite [MASK]."
Essentially you are expecting the model to prediction [MASK] as "fruit".
Each of the words above would be tokenized to its ids(more complicated than this actually), including the token [MASK], which has its special id = 103.

are we passing weights to the model to ignore prediction from tokens except "MASK".

You don't have to pass weights, for BertForMaskLM, the model handle the masking for you as long as you provide the attention_mask array, indicating which are the token that is unmasked, thus not included in loss calculation. What it does is that it basically returns "-100" as the output, and when you compute loss = CrossEntropyLoss(prediction, groundTruth), PyTorch will ignore the elements that is "-100". You can adjust this behaviour by changing the default parameter:
nn.CrossEntropyLoss(ignore_index=-100).
I'm quoting from the paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,
"we simply mask some percentage of the input tokens at random, and then predict those masked tokens".
What exactly does it mean???
Is it that the loss is calculated for masked token alone while ignoring predictions from rest of the tokens. If so what would be the right way to do this i.e. are we passing weights to the model to ignore prediction from tokens except "MASK". It will be helpful if someone can provide a link regarding same.