Fully trained models and KD Translating the effects of hyper-parameter optimization to fully trained models, we observed mixed results. Firstly, comparing optimized models with baseline implementations indicated that we were successful in creating increasingly faster models. Interestingly, KD models had around half of the baseline implementations but were similar or even faster than the fastest trained from scratch model with five times less model parameters. Another interesting finding is that speed did not necessarily translate one-to-one with decreases in carbon footprints. This is possibly due to the rather basic method of calculating carbon footprints through power usage over the inference time. Though we did observed a general trend where smaller, faster models translate to lower carbon emissions. Moreover, we also observed a substantial increase in the number of invalid smiles generated using smaller models, especially when using KD. This does indicate that smaller decoder size comes with a cost, and KD might introduce competing objectives during training, meaning that the model might prioritize similarity with the original model over correct SMILES prediction. This could be the reason why the pretrained models dominated the Pareto front, but had the worst performance during full-size training. Finally, we observed that using the last epoch for inference on USPTO-50k results in models more optimized on top-1 accuracy, where models using the optimal validation loss (around 100-200 epochs) results in models more optimized for top-10.