WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents
world model;embodied agent;neurosymbolic;large language model
Desk reject due to significantly reduced left margin.
Submission Desk Rejected by Program Chairs
@misc{
zhou2024walle,
title={{WALL}-E: World Alignment by Rule Learning Improves World Model-based {LLM} Agents},
author={Siyu Zhou and Tianyi Zhou and Yijun Yang and Guodong Long and Deheng Ye and Jing Jiang and Chengqi Zhang},
year={2024},
url={https://openreview.net/forum?id=GE0UKtI6Lf}
}
Can large language models (LLMs) directly serve as powerful world models for model-based agents? While the gaps between the prior knowledge of LLMs and the specified environment's dynamics do exist, our study reveals that the gaps can be bridged by aligning an LLM with its deployed environment and such "world alignment" can be efficiently achieved by rule learning on LLMs. Given the rich prior knowledge of LLMs, only a few additional rules suffice to align LLM predictions with the specified environment dynamics. To this end, we propose a neurosymbolic approach to learn these rules gradient-free through LLMs, by inducing, updating, and pruning rules based on comparisons of agent-explored trajectories and world model predictions. The resulting world model is composed of the LLM and the learned rules. Our embodied LLM agent "WALL-E" is built upon model-predictive control (MPC). By optimizing look-ahead actions based on the precise world model, MPC significantly improves exploration and learning efficiency. Compared to existing LLM agents, WALL-E's reasoning only requires a few principal rules rather than verbose buffered trajectories being included in the LLM input. On open-world challenges in Minecraft and ALFWorld, WALL-E achieves higher success rates than existing methods, with lower costs on replanning time and the number of tokens used for reasoning. In Minecraft, WALL-E exceeds baselines by 15-30% in success rate while costing 8–20 fewer replanning rounds and only 60–80% of tokens. In ALFWorld, its success rate surges to a new record high of 95% only after 6 iterations.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Siyu_Zhou2 ~Tianyi_Zhou1 ~Yijun_Yang3 ~Guodong_Long2 ~Deheng_Ye1 ~Jing_Jiang6 ~Chengqi_Zhang1
Siyu Zhou Tianyi Zhou Yijun Yang Guodong Long Deheng Ye Jing Jiang Chengqi Zhang
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
world model embodied agent neurosymbolic large language model
I certify that there is no acknowledgement section in this submission for double blind review.
zhou|walle_world_alignment_by_rule_learning_improves_world_modelbased_llm_agents
/pdf/376bb63d2b83ec7e43b986774f0cdf6d6f05e707.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/51982f947c5beb86e7c0c578056e9d959d7e18fe.zip
WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents
ICLR 2025 Conference Desk Rejected Submission
ICLR.cc/2025/Conference/Desk_Rejected_Submission