Despite the promising capabilities demonstrated by the current generation of LLMs, specific risks need to be mitigated for their effective application in educational settings. . Consequently, students and teachers must be aware of the possibility of LLMs providing convincing, but false, responses when prompted, and use these tools carefully with appropriate verification. This may be particularly challenging for students learning new fields, such as chemistry or more specifically advanced computational chemistry.