Although the mean bias is clearly improved, the distribution shows that a minority of the calculations were worse with Sage. An important question is what portion of the shifts are due to changes in the force field versus being due to statistical uncertainty in the simulations. To estimate the proportion of variance due to force field changes, we performed a deconvolution analysis, fitting a Gaussian distribution to the distribution of shifts and assuming the simulation error is Gaussian with 0 mean, and a standard deviation equal to the average propagated simulation uncertainty of a shift. Outlier analysis indicated that the Gaussian assumption is reasonable, with 95% of points (across both non-aqueous and aqueous Δ𝐺 𝑠𝑜𝑙𝑣 ) falling with the 2-sigma limit vs. the expected 95.4%. Using these assumptions, we estimate the percentage of variance due to force field changes to be 85% for non-aqueous Δ𝐺 𝑠𝑜𝑙𝑣 and 96% for aqueous Δ𝐺 𝑠𝑜𝑙𝑣 , with the rest due to statistical noise. This means the changes due to optimization are primarily due to shifting of force field error from one set of molecular solvations to another, but with an overall reduction in total error. Figure (data also shown in Supporting Information Table ) compares performance between Parsley 1.3.0, Sage 2.0.0 and GAFF 2.11/AM1-BCC, a widely used small molecule force field, paired with the AM1-BCC a fast charge model generally considered to be sufficiently accurate for pharmaceutical applications. In addition to calculating benchmarks on Δ𝐺 𝑠𝑜𝑙𝑣 for aqueous and non-aqueous solvents, we use the results of those calculations to calculate aqueous to non-aqueous transfer free energies (Δ𝐺 𝑡𝑟𝑎𝑛𝑠 (𝑎𝑞 → 𝑛𝑜𝑛𝑎𝑞)) for solutes that have measurements of Δ𝐺 𝑠𝑜𝑙𝑣 for water and non-aqueous solvents. Transfer free energies are a useful benchmark target for a small molecule force field because they are analogous to the process of a small molecule ligand being transferred from an aqueous bulk phase to a non-aqueous binding pocket. While Sage 2.0.0 performs slightly better on both aqueous and non-aqueous Δ𝐺 𝑠𝑜𝑙𝑣 , GAFF 2.11/AM1-BCC has the best performance for Δ𝐺 𝑡𝑟𝑎𝑛𝑠 (𝑎𝑞 → 𝑛𝑜𝑛𝑎𝑞). This seems to be due to a cancellation of error between aqueous and non-aqueous solvents in GAFF 2.11/AM1-BCC; in both environments, solvation free energies are overpredicted relative to experiment. In Sage, aqueous solvation free energies are overpredicted in aqueous solution but not non-aqueous solution. This indicates that GAFF 2.11/AM1-BCC's superior prediction of Δ𝐺 𝑡𝑟𝑎𝑛𝑠 (𝑎𝑞 → 𝑛𝑜𝑛𝑎𝑞) benefits from a cancellation of error between aqueous and non-aqueous phases. This suggests that future OpenFF force fields could benefit from the training of a companion water model.