In this work, we aim to evaluate the capability of LLMs to predict material properties, extending beyond composition-related tasks to include structural tasks from the Matbench benchmark, which, to our knowledge, have not been previously tested with LLMs. To incorporate the material structure into a LLM, it is necessary to devise a method for converting the structure into a sequence, ideally in such a way that allows for accurate reconstruction back.