what is the difference between positional vector and attention vector used in transformer model?
So the question asks between the difference between an attention vector and a positional vector.
To answer this question, will give some context into how the transformer differs from a sequential model, such as RNNs and LSTMs.
In the case of RNNs and LSTMs, data is fed sequentially &quot;one-by-one&quot; into the model to predict the output (whether that is at each time step or after observing the whole sequence. This is irrelevant though in the context of the question.)
In a transformer model, the whole sequence is fed into the model, just like as you would with a conventional neural network. However, the problem is that, unlike with RNNs/LSTMs, there is no way for the transformer model to understand the ordering of the instances in the sequence as the whole sequence is fed into the model. Therefore, we need positional embeddings (positional vector, in your terminology) to add information to the individual instances which tells the model the ordering of the instances in the sequence.
Attention, in the context of transformers, works on the basis that it assigns higher coefficients to instances in the sequences which are most relevant to decoding the hidden representation from the encoder. Unlike a basic encoder-decoder model, with attention, we are able to flexibly assign which input instances in the sequences have the most &quot;say&quot; in predicting the next output instance in the output sequence.
I hope this clarifies some understanding. If not, there is a great article on transformers here: http://www.peterbloem.nl/blog/transformers