Using a RF with only RDKit descriptors consistently results in a scoring function with greater performance than a RF with the AutoDock Vina terms, and is often close to or even exceeds the performance of RF-Score, even when test-set similar ligands are excluded from the training data. This is surprising, since we would not expect to be able to predict protein-ligand binding affinity across a diverse set of proteinligand complexes without knowing which protein the ligand was assayed with. We discuss the possible source and interpretation of this signal next, but this observation may suggest that the redundancy between the PDBbind core set and the remainder of the database makes this particular approach to training and test set construction inappropriate for validating and benchmarking machine learning scoring functions for predicting protein-ligand binding affinity.