where the sum is over the channel dimension. The above formula formalizes our intuition: for a given convolutional layer p, each channel output activation ψ p,cp (x) is weighted by its contribution ∂(ξ p,cp • Λ)(x)/∂ψ p,cp (x) to the endpoint prediction. The attribution method (1) is known as activation×gradient, which is a natural extension of input×gradient to obtain layer-wise attributions. Our implementation of ( ) is based on the captum package . Figure depicts some layer-wise explanations. We notice that attributions for early layers, as expected, focus on simple geometric features like atoms and bonds, in contrast to attributions for deeper layers, where explanations involve entire functional groups.