The ability of materials machine learning to draw upon prior work has led to steady gains in accuracy for a variety of tasks. In particular, the degree of improvement can be quantified for the field of materials property prediction from crystal structures. As we will later show, the state-of-the-art-models today can outperform those from 7 years ago by over a factor of 7 improvement in accuracy. With the remarkable gains in accuracy comes the potential to do more science with greater confidence using machine learning models. To study the improvement of structure-property models in greater detail, I make use of the matbench 53 leaderboard (). The leaderboard measures the performance of various algorithms on a series of benchmark tasks. The current top performers for each of the 13 tasks is presented in Table . The tasks are ordered by the number of samples (i.e., data points) in the task. There is a clear delineation in the leading algorithms based on the number of samples in the task. The MODNet algorithm is the clear leader on tasks with <10,000 samples, leading 5 out of 7 tasks in this category. However, these particular tasks do not have a structure provided and only chemical composition is known. Meanwhile, for tasks with >10,000 samples, the (related) coNGN and coGN algorithms lead 5 out of 6 tasks. This separation based on sample size (with different leading algorithms at that time) was observed in the original matbench paper . However, a more careful interpretation of the data shows that today, in all tasks where a structure is provided (even the phonons task with only 1,265 samples), some form of crystal graph neural network is the dominant algorithm. Algorithms such as MODNet and AMMExpress 53 come into play only in tasks in which the structure is not provided, for which the crystal graph neural networks cannot be applied at all. We next examine progress over time for one of the matbench tasks. In Figure , we plot the improvement in mean absolute error over time in predicting formation enthalpies of compounds as calculated by density functional theory simulations and tabulated in the Materials Project database. The most significant feature of Figure is the sharp drop in mean absolute error when progressing from the random forest model with Magpie descriptors to the crystal graph neural network (CGCNN). The switch from more conventional machine learning techniques with a series of hand-tuned features to that of a neural network architecture with features that are largely learned on-the-fly led to a tremendous, immediate jump in performance (from >100 meV/atom to <35 meV/atom). Indeed, for large data tasks, it is generally the case that hand-tuned features are not necessary for good performance . Subsequently, various extensions and improvements to the CGCNN architecture have steadily reduced the error further. State-the-art-models today can reproduce formation enthalpies of compounds within the Materials Project database to <20 meV/atom, which is in most cases lower than the error of the density functional calculations as compared to experiments and of various experiments amongst themselves . At this point, the matbench task can likely be considered "solved" and more difficult tasks need to be designed.