The first two reactions are taken from the USPTO-50k test dataset. The remaining three case studies originate from industrial and literature examples. The goal of the interpretability study is to uncover whether deep-learning frameworks are able to find important functional groups (or motifs) to the reaction. These functional groups should make the product thermodynamically favourable compared to its precursors and thus act as a driving force of the reaction. To ensure the success of the Explainer algorithm in identifying important subgraphs to the GNN's model prediction, the fidelity (as calculated through 20) is shown in Table . From the table, it is concluded that almost all returned subgraphs GS from the GNNExplainer are key to the model's prediction (with the exception of Test molecule 1 and Salmaterol for EGAT). Thus, a confident discussion for model interpretability is enabled. The first case study reaction tested an amide bond formation as shown in Figure . The reaction combines a secondary amine and carboxylic acid to form the respective amide and an equivalent of water. The formation of amides from these reactants is subtly thermodynamically favoured. However, reactions of this form will often require high temperatures or additives to overcome significant energy barriers. ). The EGAT model proposes a different reaction centre compared to the case study (subplot e). This reaction undergoes a carbon-carbon bond formation, which commonly involves nucleophilic carbon centres. These carbons can be produced through the use of organometallic species such as Grignard reagents (Figure ). While the Grignard reagent can be envisioned to nucleophilically attack into an N-formamide or carbamoyl chloride, it may be challenging to selectively produce the product. In literature, the carbonyl group was seen to be reduced to an alcohol during the nucleophilic attack rendering this reaction profile unfeasible for this bond disconnection . Moreover, it would likely be difficult to retain the precursor bromide species to reach this level of functionalisation. The node importance identified by the EGAT model (subplot c) highlights the oxygen on the carbonyl which does induce a slight delta positive charge on the carbon. However, since the fidelity is 0, the returned subgraph GS is not confident, rendering the node mask uncertain. Nonetheless, the amide bond formation is a well-understood and (nowadays) optimised reaction. A chemist would strongly prefer this reaction over the more difficult C-C bond formation. Thus, the GNNExplainer is queried to provide a node mask for the amide formation. The EGAT (Figure ) is seen to successfully highlight the carbonyl functionalisation as done by the D-MPNN model. Case Study 2 -Test Molecule 2 Figure shows the second reaction obtained from the test dataset, namely a Wittig reaction. The progression from reactants to products in this reaction is largely driven due to the generation of a new, strong P=O double bond. In addition, the generation of the alkene bond in the product is also slightly stronger than the precursor carbonyl bond. Given that the models are unaware of the P=O bond formation (as it is a byproduct), they must derive the bond disconnection from the smaller thermodynamic benefit of the alkene generation.