We optimized the parameters for 128 epochs with the AdamW optimizer. Following recent exploration of the parameter tuning recipes , we decayed learning rate by a cosine schedule after linearly warming up the learning rates for the first epoch. We set the learning rates for ALIGNN , MatSciBERT , and the projection heads to 10 ´3, 10 ´4, and 10 ´3, respectively, and the coefficient of weight decay to 10 ´3 in the experiments.