Neural scaling quantifies the performance improvements of large chemical models with increasing model and dataset sizes. Next, with a strategy in place to efficiently scale up experiments using TPE, we investigate neural scaling in ChemGPT and NFFs. For each model, we perform TPE to identify good hyperparameter choices that are predicted to perform well over a range of model and dataset sizes. Then, we systematically vary the dataset size (d) and model size (m) and perform exhaustive experiments to determine the converged loss, L(m, d). For efficiency and to isolate scaling behavior, we fix hyperparameters from TPE as m and d are varied, but strictly speaking the optimal hyperparameters will change as m and d vary . Due to computational resource limitations, we train ChemGPT models for a fixed number of epochs to determine the loss.