Our previous analysis of the performance of the models on predicting solvation energies in water give clues on what is important in estimating the performance of the model. We noted, in particular, that zwitterions were overrepresented in the molecules with the largest error (48% of largest errors were zwitterions, compared to only 0.4% in the test set). The poor-performance of the models on a class of molecules that are structurally distinct from most of the training set is reflective on how the MPNN models implicitly learn the behavior of function groups within molecules. Unseen or rare functional groups will have little training data, which leads can lead to poor performance. Such behavior suggests for us to consider metrics that measure similarity in functional groups between molecules to estimate the performance of our model.