Attention mechanisms are at the heart of transformers' success, providing a device to evaluate the relative importance of nodes in a graph. Traditional dot product attention consists of a procedure where the layer inputs are projected into a set of queries (Q), keys (K), and values (V) representations. Using these projections, new attentionweighted representations can be calculated using equation 1: