With only 20% of the total training budget, we are able to identify model configurations that significantly outperform the default settings from Hug-gingFace. The procedure is easily repeatable for new datasets and enables accelerated HPO. While training procedures for large language models like ChemGPT are well established, scaling NFFs to larger datasets and more expressive models requires new, scalable training procedures . Large-batch training through data parallelism is one method for accelerating training, but there are known limitations and correct batch sizes vary widely for different domains . This problem is particularly acute for NFFs, where each datapoint actually contains 3N + 1 labels for energies and atomic forces, where N is the number of atoms, creating a large effective batch size with large variance within each mini-batch. Hence, it has been observed that small batch sizes (even mini-batches of 1) work well across different NFF architectures . TPE provides a method for quickly evaluating the speed-accuracy trade off for different combinations of batch size and learning rate, which are interdependent and must be varied together to enable large-batch training.