Here, we introduced a weighted loss function for the RL process (Methods) and use this to explore down-weighting updates based on surrogate predictions relative to oracle-acquired labels (always given a weight of 1) to counteract the sensitivity of the RL-AL process to errors in the early surrogate model (Figure ). We consider weighting the surrogateprediction and oracle predictions equally (full belief), weighting the surrogate model predictions 0.75 or 0.25 to put more onus on oracle-predicted values, setting the weights for compounds to 0 where the surrogate model uncertainty is too high (> 0.025 or > 0.05) (Supporting Figure ) ,and finally using a 0 weight for all surrogate predictionsthereby updating the RL agent based on the oracle only.