When model input can be directly translated to structural parts of a molecule, explanations from XAI methods more naturally correspond to human intuition. These XAI techniques extend their importance assessment capabilities across global and local dimensions. On the global context, the evaluation centers on the role of variables in shaping the model's overall construction . In the local context, the attention shifts to the influence of variables on specific predictions, a facet that aligns with our current investigation. Local prediction elucidation strategies have emerged in the literature, including approximating complex models with simpler ML counterparts , or leveraging game theory to pinpoint influential input variables by comparing a selection of subsets . Alternatively, researchers have harnessed the model's intrinsic architecture to unveil its decision-making processes. In deep learning, integrated gradients use accumulate local gradients to assess variable effects on output. Alternatively, XAI can use inherently interpretive parts of a model. The latter has proven especially influential by using the attention mechanism of transformers, exemplified by recent work from Qiang et al. that utilize the transformer attention mechanism thought to be inherently interpretable .