Our work, specifically our findings with ChemGPT, suggests a "bittersweet lesson" for scientific deep learning -although incorporating principled physical priors and domain knowledge into scientific machine learning frameworks will continue to play an important role in this field, achieving sheer massive scale in model size and data diversity is likely to be a key component in some scientific deep learning breakthroughs. This finding presents exciting opportunities to further explore what techniques and lessons can be taken from traditional deep learning application domains. The neural scaling results indicate that traditional techniques for training large language models apply when scaling models using string-based representations of molecules. The additional dataand model-efficiency gained by careful incorporation of physical priors observed in the neural force field experiments suggests non-trivial interactions with typical scaling techniques. The trend towards larger models introduces challenges in any field, namely, increasing resource demands and environmental impact. We provide strategies and examples for scalable chemical deep learning here, but studies at massive scale remain inaccessible to most researchers. Likewise, strategies for reducing the environmental impact and carbon footprint of largescale deep learning exist, but the engineering resources and hardware needed to do so are concentrated in large organizations . However, a growing body of research suggests that large models accelerate the optimization problem and finding of robust solutions, and that once trained, techniques like model distillation, quantization, and dataset pruning are effective at reducing model and dataset sizes while retaining performance.