Where WQ and WK are projection matrices representing query and keys projections, hi and hj are the hidden representations of node I and j, d is the hidden dimension size. Spatial encodings, represented as 𝑏 𝜙(𝑣 𝑖 ,𝑣 𝑗 ) , is a set of learnable scalars indexed by the function 𝜙(𝑣 𝑖 , 𝑣 𝑗 ). In Graphormer-IR, this function is chosen to be the shortest path between two nodes in the graph (𝑣 𝑖 , 𝑣 𝑗 ), assuming that they are connected. Since Graphormer's global receptive field means that all nodes attend to all other nodes in graph, Graphormer can adapt 𝑏 𝜙(𝑣 𝑖 ,𝑣 𝑗 ) to incorporate the relationships and weightings which depend on graph structure (e.g., nodes that are closer may provide more meaning than those that are far away). This is in stark contrast to the MPNN architecture, where node/edge level information is only passed a finite number of steps away from its starting position. Graphormer also utilizes a novel edge encoding scheme (cij) where edges are considered in the correlation of node features. For each ordered node pair (𝑣 𝑖 , 𝑣 𝑗 ), Graphormer determines the shortest path between them and computes an average of the dot products of all the edge features and a learnable embedding along that path: