In recent years, the deep learning research community has devoted substantial effort in improving the expressiveness and performance of graphbased models for materials property prediction. Expanding upon the initial topological-based strategies , models have been enriched with geometric information by introducing edge pairwise distance features through radialbasis-function expansions , becoming a staple in the highest-performance models . With the area moving at an extremely fast pace, additional three-body information in the form of directional embeddings through 2D Fourier-Bessel basis attention masks , or line-graphs has recently been added to both features and graph models. A new state-of-the-art architecture has been proposed by which proposes a "nested" line-graph model that builds a graph upon another graph where the original edges become vertices and the new edges encode information describing the relationships between them. Additionally, recent work has resolved issues on the invariance of graph representation describing periodic systems (such as those used for crystals) and encoded reciprocal space and geometric information , which have in turn shown promising improvements in predictive accuracy. Most recently, self attention-based transformer models have been proposed as an alternative for materials design, due to their ability to facilitate efficient training of neural networks with vast amounts of data. Despite their recent success in capturing crystalline invariance better compared to many graph-based network models , these methodologies still suffer from long training times and large memory footprints.