FinRipple: Aligning Large Language Models with Financial Market for Event Ripple Effect Awareness
Large Language Models;Time-Varying Knowledge Graph;Event Studies;Asset Pricing
Yes
No ethics review needed.
I have the following suggestions to help improve the paper quality.

**Regarding the problem formulation**: \
1.Clearly define what constitutes a "ripple effect" in their model.\
2.Explain how the paper account for time lags between events and the ripple effects.\
3.Specify the time frame for the predictions - is it immediate impact, or do you account for delayed effects?\
4.Provide details on the temporal resolution of the data and model (e.g., daily, weekly, etc.).\
5.Clarify how long the interval is between t and t+1 in the data.\

**Regarding the related work**:\
I recommend that the authors conduct a more thorough review of recent LLM applications in finance. \
Additionally, the authors should emphasize the novelty of their approach in light of this broader literature review.\

**Regarding the input data**:\
Clarify the relationship between the news data and your KG. Are they separate inputs, or is your KG derived from the news data?\
If they are separate, explain how these two data sources are integrated into your model.\
Provide a more detailed description of your data pipeline, showing how news data and KG data are processed and used at each stage.\
If applicable, describe the process of extracting or updating your KG from the news data.
The paper proposes an interesting yet important problem, predicting ripple effects of financial market events using LLMs, the motivation is good and useful for financial scenario and applications. The problem formulation is interesting but needs to be more clear and refinement.
This paper introduces FinRipple, a novel framework that enables Large Language Models (LLMs) to analyze and predict the ripple effects of financial market events across different companies. The framework combines three key components: time-varying financial knowledge graphs (KGs) that capture inter-company relationships (like supply chains and leadership networks), integration of these KGs into LLMs using adapters as memory modules, and alignment with market dynamics through reinforcement learning using real market data. The authors demonstrate FinRipple's effectiveness through superior performance in explaining market movements compared to traditional methods, successful application in portfolio management, and comprehensive analysis showing logical consistency in explaining market reactions. This represents a significant advancement in applying AI to financial analysis, establishing a new benchmark for "event impact prediction" and offering benefits for both academic research and practical investment applications.
1. The writing is a little bit vague in some important sections, for example, problem formulation. I am little bit confused about what are you trying to predict exactly? According to the problem formulation, it seems you are trying to predict the influence of specific event on a company given a time stamp "t", using one big KG and news data. It is confusing why this is considered as ripple effects? Because there is usually time lag until some events really have effects on companies. So I want to know if you also try to predict the future influence of just present influence? And how do you define the time interval in your data? How long it is between t and t+1? 

2. The literature review is not thorough, many relevant papers in LLM for finance area are missing. Some paper I found relevant are listed below, but I believe there are more to be included.

3. It is mentioned " This task uses both the KG and real-time news data as inputs" in the paper. I am confused how is news data used separately with KG data as input? Or the KG is extracted from news data?

Yu Y, Li H, Chen Z, et al. FinMem: A performance-enhanced LLM trading agent with layered memory and character design[C]//Proceedings of the AAAI Symposium Series. 2024, 3(1): 595-597.

Yu Y, Yao Z, Li H, et al. FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making[J]. arXiv preprint arXiv:2407.06567, 2024.
Yes
N/A
No ethics review needed.
See in Weakness.

In addition:

Q1. there is limited discussion of performance during extreme market events. Could more examples of extreme events be shown over the test period? Because this period is very special (COVID-19 pandemic). 

Q2. Because the author's framework is designed to target “market ripple effects”. So, what is the frequency of KG updates? What is inference time under different backbones LLM for the proposed framework? How about the memory storage requirements for storing and updating KGs?
1. The authors discuss the impact of financial events on the predictive performance of the models. The authors combine KG, LLM, and financial theory to enable a structured approach to handling ad hoc financial market information. 

2. Use of adapter injection knowledge without retraining the full model.

3. Integrated asset pricing theories through reinforcement learning. 

4. The author provides a comprehensive experiment: a. detailed experimental setup with multiple baseline comparisons; b. Detailed ablation studies explaining the contribution of each component; c. provided detailed analysis and case study.
This paper discusses the impact of events on prediction in the financial domain. Furthermore, the author proposes the FinRipple framework, which integrates a time-varying financial knowledge graph into LLM to improve LLM's analysis performance for events. In addition, the author experimented with the portfolio management task to verify the effectiveness of FinRipple's performance.
1. The performance of the framework may depend on the construction of high-quality KGs. The author lacks a discussion on the quality of the constructed knowledge graphs.

2. More details are lacking for the construction of KGs. For example, although the author defines the relationships in KGs, how missing or incorrect data in these relationships is handled? 

3. From lines 281-289,  while the author uses CAPM residuals to measure event impact, it doesn't adequately justify why CAPM is the more suitable choice over other asset pricing models. In the appendix, the author also doesn't sufficiently explain why CAPM is more suitable for capturing event impacts.

4. From ines 292-296, there is an unclear Reward Function definition. In further, the choice of λ=0.1 (in the appendix) lacks an explanation.
Yes
I have no ethics concerns.
No ethics review needed.
please refer to the above weaknesses.
1. The task is interesting and meaningful. It is really useful to directly assess the ripple effects in the financial markets. 
2. The overall framework is novel and makes sense. It leverages LLMs and includes several novel designs to incorporate the knowledge graph as well as finetuning the whole model using reinforcement learning.
3. The experimental results are good. Multiple experiments demonstrate the effectiveness of this work.
This work aims to tackle the task of assessing the ripple effects of sudden events to the financial markets. It first formulates the problem of 
event impact prediction, and then proposes an LLM-based framework to fulfill this task. Various experiments demonstrate the usefulness of the proposed methods and results.
1. Concerns about problem formulation: 
    a. The input of the prediction model includes both KG and news data, while the equation of the problem formulation (line 231-234, PS. it’s recommended to assign an equation number for each equation for easier reference or mention) only contains G^t. Even though the news data will be first converted to graph and will not be directly consumed by the function f_{\theta}, it should be consistent between the equation (line 231-234) and the textual description (line 227-228), where the textual description says you use news data while the equation only presentes the graph.
    b. There are two types of prediction: 1) make forecast to the future based on the historical observations (extrapolation); 2) predict the effects of events by observing the events already (nowcasting or interpolation). Actually, this paper follows the second type (interpolation). However, I think it is as similar or even more meaningful to work on the extrapolation setting, where you should only use G^{<t} to predict r(c_i^t, e_j^t), or you can use G^{t} to predict r(c_i^{t+n}, e_j^{t+n}), where n is the forecasting horizon and can be set as {1,2,3,...}. Since the effects of events are continuous and evolving, it is better to focus more on the future rather than the concurrent timestamp only. More importantly, there could be a risk of data leakage issue, that the task may be just event-company relation “extraction” instead of “prediction”, because the given news articles already demonstrate such relations. 

2. Format of event: This work just uses the raw-text representation of events, i.e., news articles. However, the text-formatted events are quite long (with 5443 words in average), consequently, it increases the cost of LLMs as well as stricter requirements for long-context understanding capability of LLMs. Moreover, the raw text may be prone to noise and misinformation in practice. I am wondering if it is better to consider a certain structured representation of such events, by following some schema or ontology that predefines the event types and parameters. 

3. Direct evaluation of the knowledge graph injection: Do you have direct evaluation of the performance of knowledge graph injection, by following some knowledge graph or temporal knowledge graph prediction formulation? This can have a precise assessment how effective this module is and how much the errors induced by this module.

4. Baselines other than LLM-based methods: All the baseline methods are based on LLMs, do you have any comparison with conventional methods before LLMs, such as GNN-based methods?  

5. Typos: line 773: March 2019 to December 2021 != 226 months
Yes
No ethics review needed.
1. Lines 401-422 discuss the issue of high refusal-to-answer rates. I am curious whether the overly complex formatting requirements could be contributing to this high refusal rate by making it challenging for models to follow the specified output structure. Did the authors consider simplifying these formatting requirements (Appendix B2) to potentially reduce the refusal rate?
2. Line 861 shows that you employ a fine-grained scoring system ranging from -10 to 10. I wonder if you encountered instances where the model tended to favor neutral or less distinctive scores.
3. Recent studies in event-level financial text analysis and modeling should be cited.
This paper presents a particularly solid piece of work. By leveraging LLMs with financial KG and preference alignment techniques, the authors effectively address the challenges outlined within the financial domain. The paper presents comprehensive and detailed experiments with well-designed setups that quantitatively demonstrate the effectiveness of the proposed method, showcasing its high practical applicability in financial market analysis.
This paper introduces a framework that aligns LLM with financial market dynamics to predict the effects of market events. FinRipple leverages a time-varying financial K to provide structured market information, which is injected into LLMs through adapters. And It also employs a PPO-based approach to align model predictions with actual market behavior. Experimental results show that FinRipple outperforms baseline methods in financial tasks, indicating its potential for real-world financial applications.
1. The paper shows limited innovation in methodology. The KG-LLM integration method used in this paper is limited to employing the KG to construct datasets, without deeper integration of the knowledge graph into the model’s reasoning process. The chosen KG+LLM approach risks losing the integrity of the knowledge graph, potentially resulting in information loss.
2. The introduction frames the study’s motivation around the ripple effects of sudden events on the market. However, in the subsequent task setup and experimental analysis, the focus seems to shift primarily to analyzing events in isolation. While FinRipple indeed enhances the LLM’s capability to handle event processing, it does not clearly demonstrate its effectiveness specifically for the ripple effect problem. The introduction may be overclaiming the contributions of this work.
We propose FinRipple, a novel training framework that enhances the market analysis capabilities of large language models
@inproceedings{
anonymous2024finripple,
title={FinRipple: Aligning Large Language Models with Financial Market for Event Ripple Effect Awareness},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=o4TyewNBIB},
note={under review}
}
Event studies have been fundamental in finance, focusing on analyzing the ripple effects of sudden market events. Accurately predicting these effects is crucial for informed decision-making and effective risk management. However, the dynamic complexity of financial markets and the lack of unified modeling tools make this task challenging. Previous models, constrained by simplistic assumptions and limited scopes, have struggled to address this complexity effectively. In contrast, large language models (LLMs), with their emergent reasoning abilities, offer a promising solution. In this paper, we introduce $\textbf{FinRipple}$, a novel training framework that enables LLMs to align with market behavior and develop the capability to analyze the ripple effects of sudden events. We first construct a time-varying financial knowledge graph (KG) that is both financially meaningful and noise-reduced to accurately represent the market state. These KGs are then integrated into the LLM using adapters as memory modules. Additionally, we align the LLM with market dynamics by integrating FinRipple with classic asset pricing theories through a reinforcement learning framework. This market-alignment process collects feedback that enhances the LLM's foundational ability to analyze financial events and explain market anomalies that traditional models fail to address. Our key contributions are as follows: (1) We are the first to define the underexplored task of ``event impact prediction''. Our framework not only establishes this task but also provides an open-source benchmark, creating a unified evaluation standard for both academia and industry; (2) FinRipple complements classic asset pricing models by combining strong theoretical foundations with AI-driven capabilities, offering an enhanced analysis of residuals unexplained by traditional models. We also demonstrate its potential for practical applications such as portfolio management; (3) We conduct a comprehensive analysis to ensure that the results generated by LLMs in our framework are more logically consistent and credible, thus improving the reliability of insights for financial decision-making.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Models Time-Varying Knowledge Graph Event Studies Asset Pricing
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/d91403a022415400d10e361db4dab6a52d4026c6.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/2dbf990b32ed32040be77c4c264a7ab7ff30f1df.zip
FinRipple: Aligning Large Language Models with Financial Market for Event Ripple Effect Awareness
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission