We find that the applicability based on the molecular fingerprint MPNN yields the best performance according to both quality metrics. We find that there is the strongest correlation between both the magnitude of error and the likelihood of a 1 kcal/mol error. The MPNN-based fingerprint distance strongly outperforms a generic circular fingerprint with Tanimoto similarity, suggesting a benefit to using fingerprints trained to reflect a certain atomic property as a similarity metric. In general, we find significant need for developing good domain of applicability metrics. As illustrated by specialized MPNN fingerprints outperforming conventional Tanimoto fingerprints, the dependence of model error on inputs can require complicated models. Further research on developing complex applicability metrics or, even, measuring the performance of different metrics (e.g., Ref ) is greatly needed.