Neural scaling presents a best-case scenario for model performance with increasing resources, and allows for optimal allocation of fixed budgets, e.g., to decide whether longer training, more data, or larger models will be most efficient for improving performance. Comparing neural scaling exponents also provides a fundamental metric for measuring resource efficiency across model architectures. Investigations into neural scaling in the NLP domain have revealed general conclusions about overfitting, sensitivity to architectural choices, transfer learning, and sample efficiency . These factors are equally or more important in scientific deep learning applications, where rapid advances are being made in specialized architecture development, and it is often unclear how architectures will perform beyond the small benchmark datasets that are commonly available in scientific settings.