Improving Large Language Model Planning with Action Sequence Similarity
planning with LLM;in-context learning (ICL);action sequence
Yes
No ethics review needed.
Refer to weaknesses.
1. The proposed approach is both intuitive and shows good empirical performance as it selects exemplars based on AS similarity, providing the similar types of exemplars as the test task.
2. The empirical evaluation is extensive, spanning four PDDL tasks and a natural language planning task, and it tests the method across different base models, showcasing the robustness of the approach.
The authors propose GRASE-DC, an approach to enhance planning capability in LLMs by improving ICL exemplar selection. Unlike traditional methods relying on semantic similarity, which can lead to false positives, GRASE-DC utilizes AS similarity to identify exemplars with closely aligned plan structures. The GRASE-DC pipeline comprises two stages: re-sampling high-AS exemplars and dynamically clustering them to balance relevance and diversity, yielding more accurate planning guidance with fewer exemplars. An iterative variant, GRASE-DC*+VAL, further boosts performance.
1. The effectiveness of GRASE is highly dependent on the quality of initial plans generated by the LLM with randomly selected exemplars; poor initial plans can lead to compromised AS-based exemplar selection.
2. For setups with validator access, a baseline comparison with rejection sampling could improve the analysis. E.g., under a similar validator query budget, the validator can be used to reject the invalid plans and select the better plan generated by the approach with random exemplars.
Yes
No ethics review needed.
In Figure 3, why on some tasks the planning accuracies of GRASE+VAL are higher than those of AS for certain numbers of examples? Is there any intuition behind this observation?
- **Originality**: This paper has good originality in that it proposes to focus on action sequence similarity instead of the traditional criteria based on the semantic similarity between task descriptions when performing example selection for LLM in-context learning of planning tasks.
- **Quality**: This paper has high overall quality. Most of the steps in the proposed GRASE-DC pipeline are very clearly described and discussed in the methodology section, and Section 3 as well as the appendix also provide comprehensive empirical results and analysis to support the main claim of the paper.
- **Clarity**: This paper is generally well-written and the ideas are generally well-presented in the paper. The clarity of both the writings and the figures are good. Please see the Weaknesses section in this review for things to improve.
- **Significance**: The main intuition behind GRASE-DC is relatively simple and straightforward, but its performance gain shown in the empirical evaluations is solid.
This paper proposes GRASE-DC, which is a new 2-stage pipeline for selecting examples for in-context learning of planning tasks based on action sequence similarity. In the ‘Generative Re-sampling of Action Sequence Similar Exemplars’ stage, GRASE-DC ranks and selects ICL exemplars with high action sequence similarities. In the ‘Dynamic Clustering’ stage, GRASE-DC performs AS-based clustering to further curate improved dynamic sets of examples that strikes a good balance between diversity and relevance to further improve the LLM’s planning ability. This paper also reports comprehensive experiment results on four PDDL tasks to empirically show the performance gain brought by GRASE-DC to different LLMs.
1. In the formula on Line 141, shouldn’t there be a ‘|      |’ symbol around ‘LCAS(A_i, A_j)’? According to the previous description, LCAS(A_i, A_j) is a sequence, not a number.
2. The notation and definition of the core concept ‘Action Sequence Similarity’ should be defined more clearly and strictly in the paper. Currently some mentions of AS are a little vague and confusing.
Yes
No ethics review needed.
- Why are MLP and BPE-Proxy more efficient than LCAS, as encoding the action sequence to embeddings itself requires inference time?
- Why is Sim$_{AS}$ designed as shown in line 141, whereas based on experience LCAS($A_i$, $A_j$) should be squared?
- How is the decision made for the iterative version of GRASE-DC on whether to iterate? Since we do not have test data trajectories and answers, based on efficiency, iteration is only needed in case of errors.
- This paper breaks away from traditional task similarity and instead utilizes action sequence similarity to select exemplars for ICL, enhancing the model's planning performance, which is simple yet effective.
- The GRASE-DC shows great generalization performance on more complex tasks.
- Endeavors have been made to pursue the efficiency of the exemplar selection process.
This paper explores the action sequence as a key signal to measure the similarity between exemplars in planning tasks. Based on this finding, the paper introduces a two-stage pipeline, GRASE-DC, to select similar in-context examples while maintaining diversity and relevance. Experiments on both PDDL and natural language planning tasks show the promising performance of the proposed exemplar selection strategy. Further analysis includes OOD generalization and some options to replace AS when considering efficiency.
1. The method of using clustering algorithms to improve the relevance and diversity of the selected exemplars has been proposed by other paper [1] before.
2. The selected evaluation datasets lack some real world simulated tasks such as ALFWorld, Mind2Web, ScienceWorld, etc.
3. Though the VAL mechanism is referenced from other paper, the authors are suggested to introduce it briefly in the paper to enhance the readability as VAL appears frequently in the paper.

[1] Automatic Chain of Thought Prompting in Large Language Models, ICLR 2023
Yes
No ethics review needed.
1. How do you justify that only the sequence of actions is important for capturing the plan properties? Have you considered including intermediate states (i.e., the object states and information) in the plan?

2. Could you provide a detailed pseudocode for GRASE-DC? I am a bit confused with the steps in Diversity Clustering.
[+] The overall structure is compact and clear.

[+] The use of four benchmark domains provides convincing results.

[+] The analysis section effectively discusses the efficiency of different methods for calculating Action Similarity (AS).
This paper proposes GRASE-DC, a two-stage selection algorithm designed to identify effective and diverse exemplars from candidate plans, thereby enhancing in-context learning effectiveness. In the first stage, GRASE calculates the Action Similarity (AS) between different plans based on their sequence of actions. In the second stage, DC refines the selected exemplars using dynamic clustering based on AS, ensuring a balance between relevance and diversity. The effectiveness of GRASE-DC is demonstrated through extensive experiments, which show significant performance improvements over baseline methods.
[-] My concern is that such methods may be difficult to adopt in real-world applications. I feel that in-context learning is not the optimal solution for improving the planning capabilities of LLMs, especially considering the existing o1. In-context learning is more suitable for quickly adapting the model to output desired formats or providing it with hints to follow, rather than using very long in-context prompts to enhance the model's capabilities.

[-] The paper should at least compare some baseline approaches that use the similarity between plans. In the LLM-agent area, there are papers that employ similar ideas, using a similarity metric to select trajectories from the memory module. Implementing such a simple baseline would not be difficult and could provide a better complement to the experiments, enhancing their convincingness.
We propose GRASE-DC, an iterative exemplar re-sampling method to boost LLM planning performance using in-context learning (ICL). It achieves ~10-30 absolute points of accuracy improvement on various tasks and different LLMs.
@inproceedings{
anonymous2024improving,
title={Improving Large Language Model Planning with Action Sequence Similarity},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=tpGkEgxMJT},
note={under review}
}
Planning is essential for artificial intelligence systems to look ahead and proactively determine a course of actions to reach objectives in the virtual and real world. Recent work on large language models (LLMs) sheds light on their planning capability in various tasks. However, it remains unclear what signals in the context influence the model performance. In this work, we explore how to improve the model planning capability through in-context learning (ICL), specifically, what signals can help select the exemplars. Through extensive experiments, we observe that commonly used problem similarity may result in false positives with drastically different plans, which can mislead the model. In response, we propose to sample and filter exemplars leveraging plan side action sequence similarity (AS). We propose GRASE-DC: a two-stage pipeline that first re-samples high AS exemplars and then curates the selected exemplars with dynamic clustering on AS to achieve a balance of relevance and diversity.  Our experimental result confirms that GRASE-DC achieves significant performance improvement on various planning tasks (up to ~11-40 point absolute accuracy improvement with 27.3% fewer exemplars needed on average). With GRASE-DC* + VAL, where we iteratively apply GRASE-DC with a validator, we are able to even boost the performance by 18.9% more.
Extensive analysis validates the consistent performance improvement of GRASE-DC with various backbone LLMs and on both classical planning and natural language planning benchmarks. GRASE-DC can further boost the planning accuracy by ~24 absolute points on harder problems using simpler problems as exemplars over a random baseline. This demonstrates its ability to generalize to out-of-distribution problems.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
planning with LLM in-context learning (ICL) action sequence
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/5a1360c8c6f5a6a211ed084412abca512879066c.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Improving Large Language Model Planning with Action Sequence Similarity
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission