InvestAlign: Align LLMs with Investor Decision-Making under Herd Behavior
Alignment;Investment decision;Large language model;Supervised fine-tuning
Yes
No ethics review needed.
1) Do the authors believe the InvestAlign approach could be extended to other human behavior, such as recency bias or overconfidence, which also impact investor behavior? If so, what adaptations would be necessary? Also, are there specific limitations they foresee in applying this simplified model to unpredictable or multifactorial investment decisions in real world?

2) How about including comparisons with other alignment techniques, such as reinforcement learning from human feedback (RLHF) or using existing behavioral finance datasets?

3) Could the authors elaborate on how applicable InvestAlign is to different types of investors (e.g., retail vs. institutional) or market segments with varying characteristics? More importantly, given that the alignment was validated within a specific application (the investment context), how confident are the authors in the method’s generalizability to different financial decisions or even other high-stakes domains beyond finance?

4) What specific assumptions does the theoretical model used for data generation rely on? How might these assumptions impact the model’s alignment with real investor behavior?
The paper presents a novel approach that addresses the challenging problem of aligning LLMs with investor decision-making under herd behavior. The originality lies in creatively using *theoretical solutions* from simpler investment models that have established analytical results. Then the theoretical solution can be used to generate high-quality training data, which differs from conventional reliance on real-user data for alignment. By applying a simplified mathematical solution to a more complex decision-making process, I think this paper presents a very nice contribution, which proposes a unique, resource-efficient method for improving LLM performance in behavioral finance. This approach bridges theory-driven data generation and model alignment to LLM fine-tuning strategies in finance.

One thing that I like the most of the paper is that the authors validate the approach by demonstrating faster parameter convergence and more accurate alignment with investor behaviors from both the theoretical and empirical perspective. This theoretical basis is a very nice component. Moreover, the authors provide thorough experimental evidence, including a rigorous comparison between pre-trained LLMs, real-user data, and InvestAgents fine-tuned with theoretical data. These strong experimental results, combined with theoretical justification, highlight the quality and reliability of the proposed approach.

The paper is well-written and easy to follow. 

In sum, this work holds significant theoretical contribution and practical implications for AI applications in finance. The insights could also extend to other domains where user alignment is essential but data availability is limited, broadening the potential impact. Additionally, the method’s emphasis on optimizing parameter convergence presents a meaningful advancement for the efficiency of fine-tuning methods in LLMs.
This paper introduces a method to align large language models (LLMs) with real investor decision-making processes under herd behavior in financial markets. Recognizing the difficulty and expense of collecting real-user investment data, the authors propose InvestAlign, a supervised fine-tuning (SFT) approach that uses theoretically generated data from a simpler, theoretical investment problem that can be solved analytically. Then the data constructed from the theoretical solution provides a high-quality (synthetic) training set for LLMs, which enable them to simulate investor decisions in a cost-effective and privacy-preserving way.

Key contributions of the paper include (i) constructs a dataset based on the theoretical solution of a simpler investment model, and by fine-tuning LLMs on this theoretical data, they demonstrate closer alignment with real-user decision-making data than pre-trained models; (ii) the authors theoretically and empirically show that fine-tuning with theoretical data enables faster parameter convergence compared to real-user data; (iii) using real data collected from human user, they show that InvestAgents mimic investor decisions in both the simplified and original complex investment scenarios.

In sum, this paper proposes an efficient method framework for aligning LLMs with investor behaviors using theoretical solutions, where the authors demonstrated both theoretical and empirical success for their framework.
1. While the paper introduces an efficient method for data generation through simplified theoretical models, this approach may not fully capture the nuanced decision-making processes seen in real-world investors, especially under complex, multi-dimensional financial conditions. Relying solely on theoretical data might limit the model’s robustness. I suggest the authors consider supplementing theoretical data with smaller samples of real-user data to improve robustness.

2. The paper lacks a comparison with other alignment techniques for LLMs, particularly those that might use alternative strategies such as reinforcement learning from human feedback (RLHF) or existing behavioral finance datasets. The authors could improve the paper by adding more comparison. 

3. The paper’s focus on herd behavior is well-motivated but may limit the generalizability of its findings to other types of investor biases, such as overconfidence, or recency bias. The exclusive focus on herd behavior could make the model less generalizable. The authors could comment more on whether InvestAlign can be adapted to model other investor behaviors.

4. Still related to generalizability, it is not entirely clear how generalizable the approach is to different segments of the financial market or varying types of investors (e.g., retail vs. institutional). The simplified model may not capture the complexity of larger institutional investor decisions or differences in risk tolerance.
Yes
No ethics review needed.
Do the authors believe this approach could generally be used to create LLM agents that embody human values? To achieve this, it would be necessary to address far more complex and high-dimensional problems than the rigid ones presented in this paper. Is it feasible to transform such complex problems into simplified tasks with theoretical solutions, as proposed in this study?
The study collected data through questionnaires from real participants and utilized this data.
This paper proposes a method to instill human values into LLMs. To train LLMs on actual human values or decision-making criteria, a large amount of real human behavior data is typically required, and this work aims to address that challenge. When a complex task needs to be performed, the authors propose creating a simpler task instead, obtaining a theoretical solution, and using it for training, which they argue is more efficient than using real human behavior data.
- Although the proposed P1 (the complex, real-world task) and P2 (a simpler task for which a theoretical solution can be obtained) differ significantly from an optimization perspective, they do not appear very distinct from the viewpoint of either LLMs or humans. Thus, the claim that complex tasks can be handled by replacing them with simpler tasks with theoretical solutions seems logically overstretched.
- Additionally, the proposed task seems to focus more on assessing how well the model follows theoretically calculated values rather than instilling genuine human values. Since real humans would naturally struggle to solve the problem accurately, it is unsurprising that training on a theoretical solution yields better results.
- Consequently, I feel that the logical leaps in the authors' claims are significant, and the experimental design may not be appropriate to support these claims.
Yes
No ethics review needed.
- In constructing the SFT training dataset, the paper assumes that the probability distribution of the theoretically optimal decisions approximates a Pareto distribution, treating real user data as theoretical solutions plus white noise. Is this assumption consistently valid across different market environments? If market conditions change or investor behavior undergoes significant alterations, will this affect the relationship between theoretical solutions and real user data, thereby influencing the model's training effectiveness?
- The fine-tuning training dataset in the paper utilizes input-output pairs generated by a custom template. Which large model was employed to generate these data pairs? Were there any prompt settings for other linguistic contexts in this model? The experiments involve four fine-tuned LLM models; should there also be multiple tests conducted on the large model generating the dataset? Although the prompts provided in the paper are relatively formulaic, they may still exhibit certain hallucinatory tendencies and inaccuracies. Does the paper address strategies for mitigating and resolving these issues?
- The paper incorporates a multi-round experimental design to compare the performance of LLMs before fine-tuning with that of investment agents fine-tuned using InvestAlign. By employing various investment attributes and random seeds, the study ensures the robustness and credibility of the experimental results. This systematic approach to experimental design enhances the persuasive power of the research conclusions.
- The study employs a method for constructing SFT training datasets based on theoretical solutions, utilizing the deterministic characteristics of these solutions to enhance parameter convergence speed. This innovative approach demonstrates how effective model training can be achieved even in the absence of extensive authentic user data, thereby improving alignment with human decision-making processes.
- The paper clearly outlines the importance of understanding investor decision-making processes within the contexts of microeconomics and behavioral finance, as well as the cost and privacy issues associated with traditional data collection methods. The introduction of InvestAlign not only addresses this research gap but also provides new insights for the practical application of models in financial decision-making, possessing significant theoretical and practical implications.
This paper presents InvestAlign, a methodology for constructing training datasets by leveraging theoretical solutions to similar and simpler problems, aimed at aligning large language models (LLMs) with the decision-making processes of investors exhibiting herd behavior. The paper's experiments demonstrate that LLMs fine-tuned using these theoretical solutions exhibit faster parameter convergence and superior alignment performance when addressing complex investment issues, in contrast to traditional methods reliant on authentic user data. While the results indicate the potential of InvestAlign, the authors also acknowledge that theoretical solutions may not fully capture the complexities of real investor behavior. Future research will explore hybrid approaches that integrate theoretical solutions with real user data to further enhance model performance.
- Although the training datasets constructed from theoretical solutions offer an effective method for fine-tuning, this approach has only been demonstrated to be effective in simplified ideal scenarios, neglecting the diversity and nonlinear characteristics of complex investor behavior in real-world contexts. The numerous assumptions and simplifications inherent in theoretical models may lead to suboptimal performance of this method when confronted with authentic market data.
- The paper collects a limited amount of real user data, with only 119 samples for P2 and 90 samples for P1, which may be insufficient to support the validation of the theoretical solutions discussed in Section 3.3. Furthermore, in subsequent comparisons of experimental results, this limited dataset may not adequately represent a broader investor population and diverse market contexts, potentially introducing randomness that could impact the intuitiveness and accuracy of the results.
- While the paper emphasizes the importance of studying investor decision-making, it offers relatively limited discussion on existing related research and models in the current financial market. The lack of a thorough exploration of the differences between established methods and InvestAlign may result in an inadequate assessment of the innovative aspects of this approach.
Yes
No ethics review needed.
How sensitive is the method to the choice of the "simpler problem" used to generate training data? What criteria should be used to select appropriate simplified problems?
The paper demonstrates significant originality in its approach to LLM alignment in the financial domain. Rather than following the conventional path of collecting extensive real-world data, it cleverly leverages theoretical solutions from simpler problems to generate training data. The empirical validation is comprehensive, testing multiple LLM architectures (GPT-3.5, GLM-4, Qwen-2, and Llama-3.1) and comparing performance against both real-user data and pre-SFT baselines.
This paper proposes InvestAlign, a novel method to align Large Language Models (LLMs) with investor decision-making processes under herd behavior. The key innovation is using theoretical solutions from simpler investment problems to construct training datasets for fine-tuning LLMs, rather than relying on costly real-user data. The authors demonstrate both theoretically and empirically that this approach leads to faster parameter convergence compared to using real-user data, while achieving better alignment with actual investor behavior.
While the authors demonstrate effectiveness on two specific types of herd behavior (absolute and relative), it's unclear how well the method generalizes to other forms of behavioral biases in investment decisions.
We propose InvestAlign, a low-cost and high-quality method that constructs large-scale SFT training datasets based on theoretical models, to solve the complex optimal investment problem and also align with human's decision-making process.
@inproceedings{
anonymous2024investalign,
title={InvestAlign: Align {LLM}s with Investor Decision-Making under Herd Behavior},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=pxy5wDMnzv},
note={under review}
}
Studying investor decision-making processes under herd behavior is of great significance in microeconomics and behavioral finance. Large Language Models (LLMs) can be leveraged to assist in solving complex investment problems. However, the investment decisions generated by existing LLMs often deviate from real-user data. One method to align LLMs with investor decision-making processes is Supervised Fine-Tuning (SFT), which requires a substantial amount of real-user data that is costly to collect and raises concerns about privacy and security. In this work, we propose InvestAlign, a low-cost and high-quality method that constructs large-scale SFT training datasets based on theoretical solutions to a similar and simpler optimal investment problem, rather than the original complex one. We theoretically demonstrate that fine-tuning LLMs with these datasets leads to faster parameter convergence compared to using real-user data. By fine-tuning LLMs, we obtain InvestAgents, which align more closely with real-user data than pre-SFT LLMs in both the simple and original complex problems. This highlights InvestAlign as a promising approach with the potential to address complex optimal investment problems and align LLMs with investor decision-making processes in economics and finance.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Alignment Investment decision Large language model Supervised fine-tuning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/9eed6a599a04be704383bc475f768298cb85e618.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
InvestAlign: Align LLMs with Investor Decision-Making under Herd Behavior
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission