The most influential hyperparameter on the final training set is the training quantity (compare entries 1-4, Table ). ∆-learning dramatically reduces the size of the training set, by up to 76%, while achieving similar if not better accuracy than learning the total energy (or energy and forces). Importantly, the improvement was similar if restricted to training only on the energy differences without forces, which is expected to be a common application since atomic forces are often not stored with CSP landscapes. Compared to the training quantity, the other hyperparameters are less significant, yet tuning these parameters does offer notable improvements, particularly in the efficiency. For example, we found that a large NNP committee of 18 members does not offer significant improvement over a smaller committee of 6 members despite incurring significantly greater costs. The improved average errors with the 18 member committee are within that expected due solely to a larger committee (Figure ), suggesting the dataset chosen by active learning is not higher quality. Moreover, while smaller committees could provide adequate results they were found to generally underestimate the standard deviation (Table ) and thus the uncertainty. Similar trends of diminishing returns are observed in the other hyperparameters studied. Overall, we identify the parameters of entry 6 as the best balance between accuracy and cost and we use these settings in the following sections.