What makes a transformer a transformer?
There is not one answer to this question, but one could argue that transformers heavily rely on

transforming each input into latent subspaces of queries, keys and values in order to generate attention score
a pool of transformations of the attention vectors (multi-head) according to which models can capture richer interpretations as different sections of the input embedding can attend different per-head subspaces that link back to each input