Any comparison between transformer and RNN+Attention on the same dataset?
If you go through the main introductory paper of the transformer (&quot;Attention is all you need&quot;), you can find the comparison of the model with other state-of-the-art machine translation method:

For example, Deep-Att + PosUnk is a method that has utilized RNN and attention for the translation task. As you can see, the training cost for the transformer with self-attention is $2.3 \cdot 10^{19}$ (FLOPs) and $1.0 \cdot 10^{20}$ (FLOPs) for the &quot;Deep-Att + PosUnk&quot; method (the transformer is 4 times faster) on &quot;WMT14 English-to-French&quot; dataset.
Please note that the BLEU is a crucial factor here (not merely training cost). Hence, you can see the BLEU‌ value of the transformer superior to the ByteNet (Neural Machine Translation in Linear Time). Although the ByteNet has not adopted the RNN, you can find the comparison of the ByteNet with other &quot;RNN + Attention&quot; methods in its original paper:

Hence, by transitivity property of the BLEU score, you can find that the transformer has already outperformed other &quot;RNN‌ + Attention&quot; methods in terms of the BLEU score (please check their performance on &quot;WMT14&quot; dataset).