The expected costs of target molecules rely on the molecule synthetic difficulty and the quality of the decision maker to reach a near-optimal synthetic route. Since we used large target molecules pools and fix the molecules synthetic difficulty by filtering the molecules SMILES string length from 20 to 100, the synthetic difficulty was fair for the chemical, biological and hybrid reactions pools. Hence, we use the median expected costs of target molecules to judge the optimisation results from the three reactions pools. As the policy model 'decision makers' being trained and optimised, whilst the median expected cost of the target molecules from the biological reactions pool has a significant jump at iteration 12, those from the chemical and the hybrid reactions pools both reduce gradually over the 20 iterations (Figure ). The three curves all tend to be stable in the last five iterations, which means they all approach optimisation limits.