The advantage of our approach is that it allows each strategy to be compared on a standard basis without the need for expensive laboratory experiments. In preparing this study, we ran over 50,000 Figure : Overview of the approach used by Summit. Strategies are machine learning algorithms or baselines. Benchmarks are models of reactions that simulate physical experiments. Strategies and benchmarks are used together in an iterative approach to find optimal reaction conditions. This software framework enables standard comparisons of strategies on different repeatable benchmarks without the need for expensive experiments. Furthermore, the same framework can be applied confidently to real experimental case studies. virtual reactions, which would have been cost-prohibitive in a laboratory. To facilitate others using our workflow, we release an open-source software package for optimising reactions called Summit. Summit contains the benchmarks developed in this work and a framework for implementing more in the future. Furthermore, it includes standard implementations of seven strategies and can easily be expanded to include more. We envision that researchers will be able to use Summit to test optimisation strategies on benchmarks and, then, use the same software for optimisation of their physical experiments. This will be particularly important for emerging work on closed loop optimisation, where experiments suggested by a strategy are executed by an automated experimental setup and fed back to the algorithm. 2 Results and Discussion