Importantly, expression does not depend on η; we do not require that the reward R τ (z) is differentiable with respect to z for policy gradient estimation. We can therefore incorporate numerous molecular properties as constraints without the need for complex modifications or approximations, such as surrogate models. However, in cases where evaluating molecular properties is costly, surrogate models can be employed.