Table . Description of performance for four kinds of tasks using GPT-4, GPT-3.5-turbo and GPT-3.5-turbo with prior knowledge (GPT-3.5+PK), respectively. When the LLM is GPT-3.5-turbo, the accuracy of the agent decreases, with 37.5%, 78.1%, 84.4%, and 78.1% for database-related, molecular data structuring, Prediction & visualization, and search tasks, respectively (the overall one is 69.5%). Regarding the significantly difference of the performance for database-related tasks, it is found that in the text to SQL process, GPT-3.5-turbo failed to generate proper and complete SQL queries. The problem is in the process of executing the create_sql_query_chain function, because the performance of the function extremely depends on the LLM itself. In other words, GPT-3.5-turbo has intrinsic limitation to convert text to SQL language. Except the database-related tasks, the lower performance of GPT-3.5-turbo is attributed to two primary factors: query comprehension and reasoning ability. One the one hand, GPT-3.5-turbo's inconsistency in handling varied user expressions across tasks hindered its performance. For example, when tasked with different phrasings of the same query, GPT-3.5-turbo failed to consistently comprehend the meaning (i.e., it could understand Query_2.2 and 2.4 describing a prediction & visualization task in Supplementary Information Note 10, but fail with Query_2.1 and 2.3). The main reason for this result is that GPT-3.5-turbo has a weaker understanding of different expressions for a task than GPT-4. Therefore, it indicates that a more capable LLM enhances the agent's ability to handle same tasks expressed differently by various users, thereby improving the consistency and reproducibility of the agent. One the other hand, the capacity to handle complex tasks correlates strongly with the reasoning capabilities of the LLM, as reported in other studies . To strengthen this hypothesis, we tested the performance of GPT-3.5-turbo with enhanced prior knowledge (GPT-3.5+PK) by increasing the level of detail in the prompts (the accuracy was 84.4%, 90.6%, and 87.5% for molecular data structuring, prediction & visualization, and search tasks, respectively). For instance, when using GPT-4, the system will autonomously invoke the SubSearch tool to handle a substructure search task. In contrast, this only happens when the substructure search task is explicitly specified in the prompt, indicating the need to use the SubSearch tool when files include substructures and a list of molecules. Similar to the case of automated chemical experiments , the reasoning ability of the agents benefit from the prior knowledge. However, one exception is the case of database-related tasks in which minimal improvement is shown with prior knowledge. It is noticed that the agent selects the correct tools and use them in order. Nonetheless, GPT-3.5-turbo still struggled to accurately translate queries in natural language into SQL commands, which strengthen our hypothesis about LLMs' intrinsic limitation for this kind of tasks. Given the above analysis, the performance of ChatMolData heavily relies on the capabilities of the LLM.