The logic of the policy model 'decision maker' is as follows: it determines costs of molecules based on their functional structures and previous synthetic performance and minimises the costs of synthetic planning by selecting the overall low-cost molecules. The expected costs of the molecules were learned through machine learning. Generally at each iteration, after 50 epochs of learning, the test data outputs usually show approximately 45% scaled root mean square error (RMSE) and 65% Pearson correlation coefficient (correlation) from the test data approximations. The RMSE and correlation equations and results for all three environments and all 20 iterations are shown in SI. The RMSEs are high since we tried to learn from 2049 features out of millions of molecules, and by no means could an MLP with three hidden layers fit all the costs of molecular structures using such a simple model structure. Also, we did not expect the MLP to grasp all details from the observations, since a large portion were from trialand-error noise, which would eventually cause overfitting. However, approximately 70% correlation means that the model learned the overall relationship among molecular structure, retrosynthetic depth and the expected costs, which was promising for overall predictions.