We demonstrate the methods in two example tasks, with binary and continuous feedback. The goal in the first case is to adapt the scoring function consisting of physicochemical properties to generate molecules that score high in Quantitative Estimate of Drug-likeness (QED), with binary feedback. In the second case, we train a novel scoring component for capturing the chemist's knowledge about DRD2 activity of the molecule, based on continuous-valued feedback.