This review both looks back and looks ahead. Looking back, it examines what has enabled the field of machine learning to advance so rapidly. Indeed, about five years ago it was unclear whether the field would enter a "trough of disillusionment" or an "AI winter" . However, the development of the Transformer architecture in the computer science domain and the crystal graph neural network in the materials science domain around the same time revitalized much research and led to major advancements in performance. More recently, materials machine learning is rapidly building upon advances in natural language processing, and in particular large language models such as Generative Pretrained Transformer (GPT) models (e.g., GPT-3, GPT-4, and ChatGPT). Thus, the field has largely avoided any periods of stagnation thus far and the pace of innovation appears to only be increasing.