In this study, we investigated performance discrepancies induced by ACs in the field of CPI prediction. We established a large-scale CPI benchmark with AC annotations, named CPI2M, which substantially expands the current scope of AC benchmarks in terms of data scale, enabling a comprehensive comparison across both target-specific and general CPI prediction methods. Moreover, we developed a new deep learning-based CPI method, GGAP-CPI, which enhances CPI prediction performance by mitigating the effects of ACs. GGAP-CPI consistently outperforms other methods in both general and AC-specific metrics, leveraging advanced features such as pretrained ligand encoders, protein residue embeddings, and multi-head cross attention pooling. This method maintains its superiority across various testing scenarios, including regular CPI task, "unseen" proteins with rare samples, and limited protein information. Results on CASF-2016, LIT-PCBA, and Merck FEP benchmarks also emphasize the competitive scoring and screening power of GGAP-CPI in virtual screening. Further analyses highlight GGAP-CPI's ability to discern between AC and non-AC endpoints and offer structural insights into activity cliff-related substructures through attention visualization.