Evaluating on our ScienceAgentBench (Coding tasks in Bioinformatics/Chemistry/Geo info science/Cognitive science) just got much easier and faster! Check out our update on containerized evaluation:  

(1) Task environments are set up in independent docker containers, which eliminates potential package conflicts among different tasks and allows us to remove pip-tools, a major factor of slow evaluation previously.

(2) Users can now evaluate their agents using a single bash command and no longer need to set up their own conda environments.

(3) With multi-threading, programs for each task can be configured and executed in parallel, reducing the evaluation time to only 20-30 minutes for all 102 tasks. 

Great efforts led by awesome @YifeiLiPKU and @BotaoYu24 @osunlp!