Transformer model: Why are word embeddings scaled before adding positional encodings?
This is specified in the original Transformer paper, at the end of section 3.4:



Transcription:

3.4 Embeddings and Softmax
Similarly to other sequence transduction models, we use learned embeddings to convert the input tokens and output tokens to vectors of dimension model. We also use the usual learned linear transformation and softmax function to convert the decoder output to predicted next-token probabilities. In our model, we share the same weight matrix between the two embedding layers and the pre-softmax linear transformation, similar to [24]. In the embedding layers, we multiply those weights by √model

This aspect is not justified by the authors, either on the paper or anywhere else. It was specifically asked as an issue in the original implementation by Google with no response.
Other implementations of the Transformer have also wondered if this was actually needed (see this, this and this).
Some hypothesithed arguments (source) are:

It is for the sharing weight between the decoder embedding and the decoder pre-softmax linear weights.
It is not actually needed.
It is to make the positional encoding relatively smaller. This means the original meaning in the embedding vector won’t be lost when we add them together.

For reference, there are other StackExchange questions discussing this (see this and this).