To achieve the set goal of 5000 user responses in the study, and to ensure we covered sufficient chemical space, we considered a simple batched active learning approach. Specifically, every 1000 responses we randomly sampled a large number of pairs from the initial pool of compounds. These pairs were then ranked according to their uncertainty, as estimated by the variance of their predicted δ ij values using the the Monte Carlo dropout method with a fixed rate of 0.2 and 100 predicted samples. Additionally, to ensure that comparisons were not drawn between too many compounds belonging to similar regions of chemical space, we used the clustering strategy defined in Section 2.3 and allowed up to one comparison between any two clusters in each batch.