When investigating model behaviors on a single-and multi-step basis, we could draw five key conclusions. First, an increase the training data for the model, going from USPTO-50k to USPTO-full, showed significant increases in percentage solved, but at significant increases in both search time and carbon footprints, with similar model calls and policy probabilities. Second, single-step top-1 is not predictive of number of model calls (Figure ). In contrast, multi-step search times are more correlated with the average number of model calls per search. The number of model calls reflects how often the single-step model was queried over the retrosynthesis search. Third, top-10 accuracy and policy probability are the major predictor of the average number of model calls, with lower policy probability correlating with higher number of model calls. The exception are the template-based and KD models, where low policy probability is more associated with lower model calls, as well as the KD models with high invalid percentages where lower policy probability is also associated with lower model calls. Fourth, the percentage of searches where a molecule has a solved route is also inversely correlated with single-step top-1 accuracy. Additionally, percentage solved is associated with the number of model calls, where less calls correlate to higher percentage solved. Finally, choosing the final epoch as done ). We observe that using the lowest validation loss checkpoints, with lower top-1 accuracies, but higher top-10 accuracies result in significant increases in model calls. This is paired with increases in percentage solved, where the baseline FT-Zinc increases from 54.6 % to 84.1 % when the lowest validation weights are used.