Initially we chose the set representation layer for MRS1 (which consists only of the set representation layer after a non-learned embedding based on atom invariants) by comparing the three architectures DeepSets, Set-Transformer, and RepSet . The hyperparameters were chosen based on defaults and findings from the initial publications of each method. Based on the performance of the three different implementations of MSR1 (Table ) on the four data sets BACE, BBBP, ClinTox, ESOL, FreeSolv, and Lipo, which were chosen for their relatively small size resulting in fast learning, we selected RepSet as our set representation layer of choice.