Same procedure was repeated for all target molecules to collect residual depths and expected costs of all simulated molecules. For the former 10 iterations, the simulation of each target molecule was repeated 10 times to add randomness to the built pathways. However, for the latter 10 iterations, since all pathways were built by the trained policy models, the repeated simulation results were identical, and thus, only one simulation was required for each target molecule. In each iteration, approximately a million expected cost values of molecules at their corresponding residual depths were collected. The multiple expected costs of same molecule at same residual depth were averaged to count into the training data.