Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planning
Agentic Behavior;LLMs;LLM-as-a-Judge;Synthetic Data
Yes
No ethics review needed.
Thanks for providing the basic training configurations for SFT and DPO in Tables 9 and 10. I noticed that increasing the number of iterations in DPO training could deteriorate performance. Could you please share the learning rates used for SFT and each stage of DPO? Additionally, do you have further insights beyond what is discussed in the "Iterative Training Paradigm" section?
The paper is well-written and organized. It carefully selects 200 examples from the seed dialogues generated by the llama3.1-405b-instruct model, with human annotators evaluating them across five metrics: planning, prioritization, proactiveness, clarification, and helpfulness. The concrete measurement of each axis convincingly contributes to the construction of the final reward construction.

The experimental results show that APEC-Travel-DPO enhances agency scores and efficiency, leading to more concise dialogues with fewer interactive turns. 

The ablation study is comprehensive, covering various aspects and different hyper-parameter settings. Additionally, error analysis on 20 examples provides valuable insights into the factors contributing to low-quality dialogues.
This paper introduces  full delegation, where individuals routinely entrust their routine decision-making to agents capable of making high-quality decisions in real-world scenarios tailored to personal needs. To achieve this, the paper has designed the APEC Agent Constitution, where APEC stands for Accuracy, Proactivity, Efficiency, and Credibility. The paper assesses both the process and the final outcome, and develops APEC-Travel to actively gather personalized travel preferences from users via multi-round dialogue. The experimental results on synthetic seed dialogues demonstrate impressive performance when compared to existing baselines.
While guidelines are provided for human annotators on how to assess the dialogues, there is a lack of detailed information regarding the number of annotators involved, the process of ensuring consistency through cross-validation, and other measures taken to ensure the reliability of the final scores.

In the evaluation, although the test set comprises 1,000 examples with unique personas, there is a concern that all the seed dialogues (10k), reward construction examples (1k), and test examples (1k) are generated using the same llama3.1-405b-instruct model. This raises the possibility that the training and evaluation dialogues may still share similar distributions, except for persona variations.
Yes
No ethics review needed.
1. As far as I can tell, the proposed task is less about actual travel planning (i.e., what is the quality of the proposed itinerary?) and more about preference elicitation and inference. In the real world analogue of this setting, it seems evident that many preferences can be elicited that aren't decision-relevant. How does your approach to evaluation account for this issue?
1. I think the general idea of evaluating agents beyond accuracy is reasonable and well-motivated. There are many aspects of what makes a good user experience (e.g., did the agent provide compelling evidence for its decisions?) that aren't necessarily reflected in task success metrics

2. The experiments are thorough, easy to follow, and include several ablations. Additionally, the approach to persona construction seems reasonable and relatively well-aligned to the preferences of travelers in the real world
The paper proposes a "constitution" for evaluating the behavior of LLM agents beyond accuracy. In particular, it proposes four criteria: (1) accuracy, (2) proactivity, (3) efficiency, and (4) credibility. In a travel planning setting, the paper then uses prompting-based methods to score agent trajectories on these four categories, resulting in an "accuracy score" and an "agent score" which are combined to form a reward measure. The paper then shows a proof-of-concept that using SFT on data from a stronger model, as well as DPO, scores on this reward can be increased substantially.
1. The proposed "constitution" lacks a clear motivation for why it chooses the four principles (accuracy, proactivity, efficiency, credibility) that it does. In particular, it's unclear to me why you need all four of these principles: can an agent be accurate and efficient without also being proactive and credible? Additionally, the main experiments in the paper use a different set of principles (plan & priority, proactive, clarification), calling into question the utility of the original set of four

2. The method seems to be primarily focused on distilling the behavior of a stronger model (Llama3.1-405B-Instruct) into a weaker model (Llama3.1-8B-Instruct). As a result, I don't find the performance improvements in this paper to be very surprising, and it's not clear to me that they teach us anything new. It would be nice to have clearer takeaways from the experimental work
Yes
NA
No ethics review needed.
I have listed the questions, concerns and points required for clairty above and I am looking forward to the rebuttal during the disscussion phase.
1. **Novelty** : The concept of an "Agent Constitution" is compelling and could serve as a foundation for designing ethical and efficient agent behaviors across various applications.
2. **Empirical Validation (Quality)**: APEC-Travel demonstrates the framework’s applicability and effectiveness in generating coherent travel plans, which is evaluated through agentic scores and rule-based metrics.
3. **Personalization Significance**: The introduction of 54 empirically supported personas is a thoughtful addition to enhancing travel planning experiences.
4. **Clarity**: I find few minor details missing with gap in definition and implementation of the metrics. Although the text is well written to explain the major claims made.
The paper proposes a framework, APEC, defining criteria for good agentic behavior termed the "Agent Constitution." APEC-Travel is introduced as a running scenario, demonstrating the framework's applicability to travel planning, with agents trained using DPO to align with APEC principles. The work utilizes rule-based metrics and agentic scores for training, and synthetic data generation creates 54 personas informed by a human survey for personalization in travel planning.
The paper provides an interesting perspective on establishing agentic principles and has shown the potential for real-world impact, especially in travel planning. Addressing the points mentioned could enhance its clarity, rigor, and applicability to broader domains.

1. Validation of Synthetic Personas
    * Clarification Request: The paper lacks clarity on whether the simulated personas were validated beyond their construction from survey results. Given the random entry assignment for synthetic traveler generation, it would be valuable to know if validation methods (e.g., similarity scoring to real user behavior) were employed or considered.
2. Switch from Large to Smaller Model During “Critical Persona Entry Selection”
    * Justification Needed: During Seed dialogue generation and agentic scoring, a larger model (Llama3.1-405B-Instruct) was used, likely for accuracy. However, during the "Critical Persona Entry Selection," a smaller version (Llama3.1-8B-Instruct) was utilized. Is there a performance or computational rationale for this model switch, and how does it affect outcome consistency?
3. Efficiency Definition and Mapping to Experimental Context
    * Clarification Request: The definition of efficiency as "accuracy gain per round" does not seem to align with the context described before the experiments, where it mentions "checking whether the agent has obtained critical information with limited dialog exchanges." Was efficiency calculated by invoking the Stenographer after each round to validate information retrieval or solely by accuracy differential per round? Additional clarification on this could help contextualize the metric’s relevance.
4. Exclusion of Efficiency in Reward Calculation
    * Inquiry: Why was efficiency, a key metric, excluded from the reward calculation for DPO training, despite its apparent relevance in performance benchmarking? Is it because its derived from Accuracy?
5. Sentence Construction in Section 4.3
    * Suggestion: Minor improvements in sentence structure are needed just before Section 4.3. For example, "suggesting the efficacy of our training recipe in consistently train agents that align with the principles..." could be improved.
6. Alpha Value in Reward Calculation
    * Clarification Request: The value of alpha in the reward calculation appears significant, yet it's unclear what alpha was set to for the primary results shown in Table 2. This information would aid reproducibility and could be noted in the table or text if it was overlooked.
7. Alpha Setting in Iterative Training Paradigm
    * Inquiry: In the ablation study, "Iterative Training Paradigm" shows only accuracy improvements between fixed and recursive paradigms. Was this improvement due to a high alpha value in the reward calculation? Further details would clarify the choice and impact of alpha in iterative training.
8. Evaluation Across Domains and Real-World Datasets
    * Suggestion: While APEC demonstrates effectiveness in APEC-Travel, additional evaluation on other agentic domains (e.g., web exploration, embodied agent tasks) would strengthen claims of generality. Additionally, employing established travel planning datasets (e.g., TravelPlanner[Xie et al. 2024b]) could enhance validation beyond synthetic data. I assume this as a limitation of this work.
We propose APEC Agent Constitution, a list of principles for good agentic behaviors. Following this, we develop APEC-Travel, a travel planning agent that proactively extracts hidden personalized needs via multi-round dialog with travelers.
@inproceedings{
anonymous2024towards,
title={Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=dePB45VMFx},
note={under review}
}
How are LLM-based agents used in the future? While many of the existing work on agents has focused on improving the performance of a specific family of objective and challenging tasks, in this work, we take a different perspective by thinking about full delegation: agents take over humans’ routine decision-making processes and are trusted by humans to find solutions that fit people’s personalized needs and are adaptive to ever-changing context. In order to achieve such a goal, the behavior of the agents, i.e., agentic behaviors, should be evaluated not only on their achievements (i.e., outcome evaluation), but also how they achieved that (i.e., procedure evaluation). For this, we propose APEC Agent Constitution, a list of criteria that an agent should follow for good agentic behaviors, including Accuracy, Proactivity, Efficiency and Credibility. To verify whether APEC aligns with human preferences, we develop APEC-Travel, a travel planning agent that proactively extracts hidden personalized needs via multi-round dialog with travelers. APEC-Travel is constructed purely from synthetic data generated by Llama3.1-405B-Instruct with a diverse set of travelers’ persona to simulate rich distribution of dialogs. Iteratively fine-tuned to follow APEC Agent Constitution, APEC-Travel surpasses baselines by 20.7% on rule based metrics and 9.1% on LLM-as-a-Judge scores across the constitution axes.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Agentic Behavior LLMs LLM-as-a-Judge Synthetic Data
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/0a66b3130a3c5577c002ddd79ca386a8258763fd.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission