What is "position" in CNN (im2latex) for Positional Encoding? I'm trying to build a model that maps images of math
formulas into LaTeX markup.
I found an acticle (https://arxiv.org/ftp/arxiv/papers/1908/1908.11415.pdf) that proposes an encoder-decoder architecture that solves this problem.
In encoder after the work of CNN it's proposed (page 3) to add following signals to generated feature maps &quot;in order to retain the spatial locality information&quot;:
$$PE(x,y,2i)=sin(x/10000^{4i/D})$$
$$PE(x,y,2i+1)=cos(x/10000^{4i/D})$$
$$PE(x,y,2j+D/2)=sin(y/10000^{4j/D})$$
$$PE(x,y,2j+1+D/2)=cos(y/10000^{4j/D})$$
Here the authors refer to positional encoding part of Transformer model &quot;tailoring the 1-D positional encoding technique to 2-D&quot;.
In original Transformer model (from article &quot;Attention Is All You Need&quot;) the position encoding is calculated like this:

Where &quot;pos&quot; is just an index number of word in given sentence (correct me if I'm wrong).
And here comes the question: what is &quot;x&quot; and &quot;y&quot; in first formula?
In the original transformer model, one input 'pos' was required as input was 1 dimensional (words). With encoder being CNN, the input feature vector will be 2D, so x and y are just the indexes of an 2D feature vector. By using those indexes, PE functions can produce those positional encoder images that you can find in the paper.