Random sampling brought noises to the decision-making problem, which randomly selected over all reaction options, and made it possible to explore over the decision space. Using the trained policy model, the expected costs of all reactants based on their molecular fingerprints and residual depths (discussed below) were predicted for each reaction option. The policy model 'decision maker' selects the candidate reaction 𝑟 ) , which has the minimum sum of predicted expected costs of all reactant molecules (Eq. 5). Essentially, this means the policy model would understand the expected costs of molecular structures after exploring the reaction space, and always point to the molecule structures that are easier to synthesise as the following steps.