Benchmarking of optimization algorithms has also received significant attention, with various frameworks having been established, like Coco 61 for continuousparameter global optimization, OpenAI Gym 62 for reinforcement learning, Sherpa 63 and Optuna 64 for machine learning hyperparameter optimization, and Pygmo 65 for massively parallel optimization. Similar benchmarking frameworks, yet tailored to chemistry and materials science applications, have also been proposed. Summit 66 and EDBO+ are both concerned with the model-based optimization of chemical reactions. Olympus has attempted instead to cover a broader set of optimization tasks and algorithms. Both Olympus and Summit use probabilistic machine learning models to learn continuous parameter-measurement relationships from experimental datasets. This approach allows one to obtain simulated experimental measurements that can be used to benchmark optimization strategies. Without the burden of executing actual experiments, one can collect the necessary statistics to evaluate the expected performance of different algorithms.