where N (i) refers to all edges originating at node i, the operations are performed per-element, and α ij is a 1x1x4 tensor. The softmax guarantees that k∈N (i) α ik = 1 such that the attention scores associated with each head can be interpreted as mixing probabilities along each edge. Finally, the inputted 512x1 node feature vectors, n ′ i , are reshaped into 128x1x4 tensors, n ′,h i , where each row vector plays the role of a value in the attention mechanism.