How to estimate missing values when calculating NDCG
Movie 2 get second best score but it has not been rated by user although it's highly relevant for user A based on other user ratings.

The issue with this is that you're injecting your opinion into what items are relevant to user A.

How NDCG@3 should be calculated in this example ?

You cannot evaluate your model on items the user has not rated. Generally with explicit feedback data you want to ignore missing values. Imputation may be possible, but it is more challenging and not commonly used. The correct modeling pipeline is to first make a training &amp; testing set. Train your model on the training set and then for each user rank their known items that are present in the testing set (source: p.g. 24). Then you can compute various cutoff-sensitive metrics.
In your example, you should not be predicting ratings for movie 2 or 5. If your training set was movie 1 &amp; movie 3 then you would make predictions for movie 4 -- obviously in a real application you should have more data so that there is more than one example in your test set.