We performed benchmarking on the test set described in Section 2.5 for OpenFF 1.0.0, the simulation-only optimization against the "pure only" set, and the 5 multi-fidelity optimization runs with 𝑁 = 10 initial points. The benchmarking set is described in Section ?? We focused on the 𝑁 = 10 runs because they generally had better objective function performance compared to the 𝑁 = 5 runs, and they also had larger parameter changes, meaning that they would be more susceptible to overfitting. RMSE statistics for all of these force fields are plotted in Figure . These results highlight the need to test for transferability, as run 3, which had the lowest objective function over the training set, performs worse that OpenFF 1.0.0 in three of the four physical property data types in the test set. This is caused by the very large changes in the [#8X2H1+0:1] (hydroxyl oxygen) parameters, as this force field performs worse. Similarly, run 4 performs poorly on the test set after significant changes to the [#8X2H0+0:1] 𝑅 𝑚𝑖𝑛∕2 parameter.