Model Architecture: Transformers. We choose Graphormer as our architecture because of its success in a variety of tasks and have been shown to outpace MPNNs, which have previously been the state-of-the-art model for many (bio)cheminformatics tasks. Transformers have been very successful because of their ability to describe relational data with highly contextual learned aggregations of the input representations. Attention mechanisms are largely responsible for this expressiveness, where models can assign learned degrees of relative importance between input tokens (atoms, for our purposes) in the input sequence. These higher order relationships between tokens are aggregated to generate dense representations with highly contextual of long-ranged interactions. While used with great success in natural language processing (NLP), transformers were long thought to perform poorly when describing non-linear, graph structured data.