The entire workflow is realized through the collaboration of three GPT agents, as shown in Figure . The CRESt interaction layer uses the OpenAI GPT-4 model with function call capability. It is responsible for direct voice interaction with users and, at appropriate times, calls upon the SEM agent and relay natural language instructions from the user. The SEM agent, which is also based on the GPT-4 model and forms the core of the architecture, begins operations on Phenom Pharos, a Python-controlled SEM, upon receiving user instructions. After automatic focusing and adjustments in contrast/brightness, the SEM agent transmits the captured SEM images to a vision-capable agent (expected to be integrated within SEM agent in the future, when OpenAI API permits). The vision agent extracts information from the images, processed with the help of Set-of-Mark (SoM) , and provides next-step SEM operational suggestions back to the SEM agent. This cycle iterates until the vision agent determines that the current SEM image fulfills the user's objective, at which point the SEM agent presents the final SEM image and a summary report to CRESt. This autonomous SEM still has significant room for improvement in both functionality and performance aspect, but our preliminary experiments have yielded some fascinating insights. Firstly, we were very impressed to see the default GPT-4V can indeed identify martensite region without any additional information beyond the mention of this term. We anticipate that with more specialized data provided during fine-tuning, this SEM agent can be seriously deployed in practical research production. Secondly, the integration of the SoM module significantly enhances the result from visual agent. Prior to the SoM layer addition, LLMs could only imprecisely refer areas in SEM images, such as "upper right" or "bottom left," making it difficult for the SEM agent to accurately navigate to the desired region of interest (ROI). After the SoM layer is introduced, shown in Figure , the visual agent can now explicitly refer the region under discussion. However, caution is needed when using SoM. We observed that relying solely on SoM-processed images hinders the GPT-4V's ability to identify the martensite region. After careful analysis, we realized that the GPT-4V identifies the martensite region based on a description "needle-like", which worked great on original SEM image. Yet, with SoM-treated image, it will try to find the region that has a needle-like boundary (e.g. Figure region p), instead of needle-like morphology inside of the region (e.g. Figure region a/d/g). This issue got resolved when both the original and SoM-processed images are provided. Lastly, we explored the SEM's quantification ability. One of our expectation for the visual agent is to estimate the coordination of ROI so that the SEM agent can center ROI and zoom in. Initially, we followed the mindset of us human, which was to estimate the distance based on the scale bar provided in the bottom left corner, but the results were never satisfactory. Surprisingly, we found that calculating distances based on fractions of the field width (to visual agent it's the width of the whole image), e.g. "one fourth of the FW," effectively addressed this challenge.