Why should duplicates compounds be removed from compound libraries before screening/activity prediction?
Let's consider a data set with three attributes for each compound: a chemical name, a CAS number (unique identifier for a compound) and a single empirically measured numerical interest, such as efficacy in lowering blood pressure in rats. If you had duplicate entries for a single experimental result, that could potentially bias the analysis. If you had two results for the same compound from two different experiments or measurements, you would surely want to include both in the analysis. If you had two results for the same compound (same CAS number) but named differently, you would want to keep them both in your data set, but use only the CAS number as the nominal variable for structure.