The "unreasonable effectiveness" of deep learning in domains such as computer vision (CV) and natural language processing (NLP) relies on the ability of deep neural networks (DNNs) to leverage ever-increasing amounts of compute, data, and model capacity. Large-scale models, including Bidirectional Encoder Representations from Transformers (BERT) and DALL-E , have been so successful at synthesizing information from large datasets via selfsupervised pre-training and performing a variety of downstream tasks with little to no finetuning that most state-of-the-art (SOTA) models in NLP and CV are adapted from a small set of large, pre-trained models . Naturally, we might expect that massive model and dataset scaling will be a prerequisite to achieving out-sized success for deep learning in science. Recent work such as AlphaFold , the Open Catalyst Project , and ChemBERTa indicate that larger datasets and models, pre-training, and self-supervised learningall key ingredients in CV and NLP -unlock new capabilities for deep learning in chemistry. However, unlike in CV and NLP, the path to scaling deep chemical networks and the potential benefits are unclear. Chemical deep learning can incorporate physics-based priors that may ameliorate the steep resource requirements seen in other fields . Moreover, because of the heterogeneity and complexity of chemical space and molecular machine learning (ML) tasks , training general and robust models that perform well on a wide variety of downstream tasks remains a pressing challenge . The enormity of chemical space and heterogeneity of these tasks motivates investigations of large-scale models in chemistry, because such models are wellsuited to unlabeled, multi-modal datasets . Recently, neural scaling laws have emerged as a way to characterize the striking trends of improved model performance over many orders of magnitude with respect to model size, dataset size, and compute; however, these experiments require immense computational resources and rely on well-known, domain-specific model training procedures that do not apply outside of traditional deep learning application areas.