Graphormer was chosen as our model architecture to address the known limitation of the MPNN architecture and because of its recent success across a wide variety of do-mains. .A complete depiction of our model architecture is present in Figure . Transformers are a type of encoder that are successful because they learn contextual relationships between input embeddings which are aggregated to a learned description of the input. This is achieved using a self-attention mechanism, where the model assigns varying degrees of importance to between different items in the input sequence. These importance values can then be utilized to generate contextual hidden representations which incorporate long-range contextual dependencies between items. Transformers are most well known for their applications in Natural Language processing (NLP), where the attention mechanisms and grasp of context allows models to parse the recombinant meaning of sentences and paragraphs. Before the advent of Graphormer, transformers were thought to perform poorly on graph structured data, despite their broad success in NLP tasks as embodied by the BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) models.