Our main conclusion is that the Crippen logP model provides an excellent benchmark for heatmap approaches, especially if the ground truth heatmaps can be adjusted to reflect the molecular representation. While this is straightforward for a FP representation, it is not immediately clear how to do this for graphbased representations. In any case, we have shown that a combination of a relatively simple and widely used ML model and attribution method can provide heatmaps that are in good agreement with the ground truth and give a great deal of insight into how the model has learned. Like the simpler attribution benchmarks for classification tasks that have come before it, this work sets the bar for regression tasks. : overlap distributions for a) the atomic-attribution vector and the ML vector (green) compared to the overlap between the atomic attribution vector and the null-model vector (blue) and b) the FP-attribution vector and the ML vector (magenta) compared to the overlap between the FP-attribution vector and the zero-model vector (orange). The null-model vector is created by simply assigning the mean atom contribution to each atom based on the predicted logP; for positive logP predictions all atoms will have the same positive contribution and for negative logP predictions all atom will have the same negative contribution.