We used ChemBERTa's PubChem-10M dataset for pre-training. This dataset includes 10 million SMILES strings, which represent the structures of various molecules. By leveraging this extensive dataset, we ensured that our model had a robust understanding of a wide range of chemical structures, thereby enhancing its predictive accuracy and performance in identifying drug-like molecules.