Here, we introduce a contextual message passing (COMP) scheme to sidestep the overhead of angular and other local environment embeddings by cheaply "attending" to the environment of each vertex, in-model. In particular, we modify the message creation stage and instead of only passing implicit information about neighbouring connections based on previously passed messages incorporated into their states, we instead contextualise the message function by further integrating edges from both the local and global cliques of each node directly, emulating thus an attention mechanism that captures both local and global structures of the graph and thus more complete representations of the chemical environment itself. Figure Each contextualised pairwise message is treated as an edge-update function as follows