What is "position" in CNN (im2latex) for Positional Encoding?
In the original transformer model, one input 'pos' was required as input was 1 dimensional (words). With encoder being CNN, the input feature vector will be 2D, so x and y are just the indexes of an 2D feature vector. By using those indexes, PE functions can produce those positional encoder images that you can find in the paper.