Thallium Flux Assay Data. A high-throughput ion channel screen was developed and validated at National Center for Advancing Translational Sciences -NCATS (formerly known as the NIH Chemical Genomics Center) as a modified version of the FluxOR TM thallium flux assay that detects inhibition of the hERG channel by measuring flow of a surrogate ion, thallium. The study compared the activities of 10 common hERG inhibitors measured in thallium flux and patch-clamp experiments and concluded that the homogeneous high-throughput assay can be used as a cost-effective alternative to patch-clamp technique. More recently, NCATS reported a collection of 4,323 compounds screened in thallium flux assay used for generating support vector classification models of hERG channel inhibition. In the present study, we aimed to merge this dataset with the high-confidence activity data obtained from ChEMBL database for development and critical assessment of modern machine learning approaches. First, we analyzed the correlation between flux assay data and ChEMBL data that originated from multiple assay types (e.g. electrophysiology, ion flux, radioligand binding, fluorescence, etc.). A subset of 86 approved drugs with activity data from both sources was identified for this purpose. However, since ChEMBL data spans multiple assay types, the correlation analysis was performed twice, first with patch-clamp data alone (Figure ) and next with data obtained after merging activities from different assay types (Figure ). We noticed that the outcomes from different assay types correlated well with the patch-clamp data, suggesting that the data could be used together. Furthermore, given the concordance between ChEMBL data and flux assay data, we decided to merge the two datasets to generate a combined dataset after removal of duplicates. for training the classification models. Additionally, a collection of 840 compounds was selected from an in-house library to generate a test dataset for prospective validation of the classification models. These compounds were measured in the same thallium flux assay to generate IC50 data. We did not use a single activity threshold to discriminate blockers from non-blockers in ChEMBL whole-cell patch clamp data. Previous studies using a binary threshold (1𝜇M and 10 𝜇M) provided superior performance as compared to using a single threshold (1𝜇M or 10 𝜇M). For the thallium-flux assay, a threshold of 30𝜇M was used considering the average fold difference in the activity for compounds with data available from both sources. Finally, whole-cell patch clamp hERG data was manually extracted for 177 FDA approved drugs (2012 to 2018) from their pharmacological and safety reviews. Many previously published studies do not include such data into their training or validation sets. We believe that validation on this data offers a more realistic evaluation setting for our models. Since it is hard to determine the hERG liability of a drug based on the IC50 value alone, without the knowledge of the peak serum concentration unbound to plasma proteins, we use both activity thresholds (1𝜇M and 10 𝜇M) to classify the drugs as blockers and nonblockers. Table summarizes the datasets used for modeling and validation. Five different sets of molecular descriptors were calculated and used in combination with different methods in this study. Morgan fingerprints (1024 bits) and RDKit descriptors were calculated using the RDKit toolkit. Recent progress in deep learning facilitates the development of the different molecular representations such as latent vectors that are used as descriptors for modeling molecular properties. In this study we utilized several different approaches to generate fixedlength latent vector representations for our modeling sets. The detailed methodology involved in generation of latent vectors is described in the next section. RNNs that learn on sequential data such as SMILES as input have been employed for molecular property prediction. Three variations of SMILES including the commonly known canonical SMILES, randomized SMILES and DeepSMILES were employed as input for RNN models in this study. While the RDKit toolkit was used to generate canonical SMILES, we relied on their original implementations for randomized SMILES and DeepSMILES.