Firstly, LLMs are prone to producing outputs that are factually incorrect or logically unsound . LLMs lack the capability to distinguish between the plausibility of different patterns observed in their training datasets. This problem can be addressed in several ways: ensuring good dataset design, appropriately limiting the scope of the LLMs usage, setting reasonable expectations for LLM output, focusing on outputs that are rapidly human-verifiable, and relying on more concise outputs. This illustrates potential risks in tasks like directly printing hundreds of atoms' coordinates into CIF correctly.