While proteochemometric models are widely used to integrate protein and ligand data simultaneously, the standards for their development and evaluation are often overlooked. Our focus on dataset curation, splitting, and embedding validation highlighted two main areas of concern. We assembled a comprehensive dataset of kinase-ligand interactions from sources like ChEMBL and BindingDB, adhering to stringent curation standards 3 . Our study highlights that models performed poorly with the most rigorous data splits (leaving one family out during training), indicating a lack of generalization to unseen kinase families regardless of the featurization method or use of decoys.