Furthermore, the methods of attention maps and Grads were performed using the last layer and averaged over all attention heads. Some researchers have suggested to use specific layer and head combinations to explain the model based on heuristic approaches, such as . We leave such analyses to future papers and stay consistent here with the methods from Qiang et al. . The methods of IG and SHAP were implemented using the standard Captum library and can therefore serve as proper baseline implementations.