it. 1 it. 2 it. 3 it. 4 it. 5 QMUGs Fig. | Iteratively biased generation of molecules toward large HOMO-LUMO gaps using GPT-3 finetuned on the QMugs dataset of draws. We start by fine-tuning GPT-3 on a sample of the QMugs dataset and use this to query for around 1000 gaps from a normal distribution with shifted mean (mean 4.0 eV, standard deviation 0.2 eV). We then iteratively select the high-gap samples of the generated molecules and fine-tune the model on this data (i.e., starting from the second generation, the model is fine-tuned on molecules it itself generated). Smooth curves show kernel-density estimates; the plot is truncated at 10 eV, but the models also generate some molecules with larger HOMO-LUMO gaps. We chose a comparatively large number of evaluations for this figure to increase the clarity of the visualization. For the initialization, we evaluated 2162 compounds using xTB, followed by 1670, 250, and 1572. If we limit the number of quantum chemistry evaluations to or lower 100, we can still successfully shift the distribution as shown in Supplementary Fig. .