The results from this benchmarking demonstrated the potential of using LLMs as code assistants to implement ML models for chemists (Figure ). For task 1, which involved training ML models, the LLMs demonstrated a high degree of competency (Table ), with code generation accuracy frequently surpassing 90%. This indicates a strong understanding of the ML frameworks and the ability to apply them correctly to chemical datasets. In task 2, the LLMs faced the more complex challenge of optimizing chemical synthesis conditions (Table ). Here, the more advanced LLMs (e.g., GPT-4o) showed impressive adaptability, with a success rate in over 60% of the trials, highlighting their potential to handle complex, contextdependent coding tasks. Task 3 tested the LLMs' ability to comprehend and apply unfamiliar Python packages for yield optimization. Here, the LLMs proved to be proficient learners, quickly adapting to new documentation and examples to produce ready-to-use code (Table ). Finally, in task 4, LLMs were tasked with generating executable scripts for liquid handling robots based on the suggestion made by Taks 2 or Task 3. This task demonstrated the practical applicability of LLM-generated code in automating physical processes in the lab, consistent with the previous literature findings , with successful execution reflecting the LLMs' capacity to integrate digital and physical workflows effectively (Figure ). Furthermore, we introduced a "self-reflection" mode in the benchmarking process and evaluated the codes generated with or without this model (Figure and Figure ). When an LLM's generated code failed initial execution, the error message was automatically sent to the LLM, prompting it to modify its output in real-time. It was observed that, for all models, regardless of the size and if it is open-source or not, this iterative process significantly enhanced the quality of the generated code by reducing the hallucinations (Figure ), proving that LLMs can learn from their errors and improve subsequently generated codes.