A generation cycle with REINVENT completes (i.e., returning the final set M r ) once a pre-defined number of optimization steps is reached. In a classical setting (i.e., without active human intervention), s(x) is assumed to be static throughout R generation cycles. In our setting, we describe how s(x) is adapted at every generation cycle after fine-tuning the property predictor with human feedback. This procedure is referred to as the "REINVENT Loop" in Figure .