The OpenFF Evaluator framework is a flexible, scalable and highly extensible framework for curating data sets from large, open data sources and estimating those data sets of physical property measurements and their derivatives with respect to force field parameters for optimization. The framework can use a range of common force fields, as well as an expandable range of estimation techniques. Through integration with optimization engines such as ForceBalance, the framework readily facilitates the training of new force fields directly against physical property data, as well as assessing such force fields against even larger data sets.