LLM-guided spatio-temporal disease progression modelling
disease progression modelling;LLM;spatio temporal modelling
Yes
No ethics review needed.
### Questions

1. **Handling of Repeated Measures**:  
   - "Authors mention that they leverage longitudinal population-data. How do you handle repeated measures in the training process?"

2. **Ratio Specification for Mixing LLMs**:  
   - "How do you specify the ratio (0.865:0.135) for mixing the two LLMs? Could you provide a rationale or any empirical basis for choosing this specific ratio?"

3. **Cross-Validation for Robustness**:  
   - "A simple splitting of the 255 subjects into train-test-validation is not enough. A fold cross-validation (train-test-validation) splitting multiple times would produce more powerful results. Could you comment on the reason for choosing a single split rather than cross-validation?"
### Strengths

1. **Innovative Use of LLMs in Disease Progression Modeling (DPM)**:
The paper introduces a creative and potentially impactful approach by leveraging large language models (LLMs) for graph-based disease progression modeling  in neurodegenerative diseases.
This paper presents a novel disease progression model for neurodegenerative diseases that leverages a mixture of large language models (LLMs) to infer brain region interactions for predicting temporal dynamics. The approach uses prompts to query LLMs about potential interactions between brain regions, which are then used to construct a graph that informs disease progression
### Weaknesses 

1. **Vague Language and Disorganized Manuscript**:  
   The paper uses terms like "causal interactions" without justification. The use of graph learning alone does not establish causation, as this typically requires specific assumptions or experimental design elements, such as interventions or causal inference techniques. Without these, the inferred relationships are associative rather than causal. Also,  the term "long-term" is not clearly defined, despite its critical importance in temporal modeling of disease progression. The lack of a precise timeframe (e.g., years or decades) makes it difficult to understand the model's intended scope and applicability. A clearer definition of "long-term" would strengthen the paper. Additionally,  there are instances where equations or concepts are referenced before they are defined, disrupting the logical flow and making it difficult for readers to follow. For instance, equation 9 is mentioned in the text (Section 2.2) before it is officially introduced (Section 2.5.1). Finally, there are several typos throughout the manuscript. Minor notice, but important, evaluation measures like SSE should be introduced first and then utilized with abbreviation for more clarity.

2. **Lack of Comparisons with Baselines and Limited Qualitative Results**:  
   Although the authors cite several relevant baseline methods, such as Bellot et al. (2021), they do not provide empirical comparisons. Without benchmarking against these established approaches, it is difficult to assess the advantages of the proposed method. 
Also, there are no qualitative results with predicted trajectories that would showcase that the method works in several test examples.
Additionally, in section 3.3 and 3.4 of the results authors use only the Claude3.5 LLM. However, their proposed method is the mixture of the LLMs and not only a single LLM (Claude3.5). This inconsistency, weakens result's coherence  as well as the validity of the proposed method. 

3. **Questionable Prompt Strategy for LLMs**:  
   The approach relies on querying LLMs with prompts for each brain region to infer relationships, which raises several concerns:
   - **Scientific Validity**: General-purpose LLMs lack domain-specific scientific knowledge and are prone to hallucinations. This means that the generated weights or connections could be inaccurate or even fabricated.
   - **Scalability**: The strategy of creating prompts for each brain region does not scale well for fine grained ROI brain representations, making it computationally intensive and potentially inconsistent.
   - **Stability and Consistency**: Since LLM responses can vary with prompt phrasing, it’s unclear if these outputs are reliable across prompts. An analysis of prompt sensitivity or consistency checks would be necessary to validate this approach.

4. **Limited Longitudinal Data for Temporal Modeling**:  
   With only 254 samples from 185 subjects, it appears that most subjects have only a single acquisition (baseline) and one follow-up. This data limitation raises doubts about the model’s ability to learn meaningful temporal dynamics. The authors do not address how the model captures progression trends with such limited longitudinal data, leaving the temporal component of the model questionable.

5. **Limited Discussion of Practical Application and Real-World Feasibility**:  
    The paper lacks a detailed discussion of how the proposed Disease Progression Model could be applied in real clinical or research settings. The reliance on prompts and general-purpose LLMs may hinder broader adoption, as access to specific LLMs, prompt engineering, and technical resources may not be feasible in clinical environments. Addressing these practical challenges and offering guidance for real-world application would enhance the paper's relevance.
Yes
No ethics review needed.
What is the essential motivation of employing LLM in modeling AD progression?
First try of LLM in understanding disease progression.
This work presented a proof-of-concept approach to model Alzheimer's disease (AD) using LLM. The idea is interesting. However, the feasibility and power of LLM in understanding the etiology of AD is not clearly demonstrated. Our current understanding on disease mechanism is still elusive in the AD field. Not sure how the proposed approach could advance our current understanding on AD. 

Another main concern is the data sample size. Neuroimaging data from ADNI is used in this work. There is only couple hundreds (<300) subjects having both modalities (structural connectome and pathology data). It is not clear how to address the issue small sample size.

Also, the data pre-processsing is not clear. Which brain atlas is used? Based on the node number, looks like the Desikan-Killiany Atlas. Please note this atlas does not include sub-cortical regions, means it is not a perfect brain parcellation for studying AD.

Result interpolation lacks biological underpinnings and supporting evidence from current literatures in AD.
Not clear how this approach distinct to other knowledge graph approaches. 
Lack of a killer application supporting the motivation why we need invent the new wheel for AD.
Figures are not very helpful to understand the method.
Yes
No ethics review needed.
Please refer to weaknesses.
This paper explores the use of Large Language Models (LLMs) as expert guides to enhance graph inference by integrating expert knowledge.
This paper introduces a novel spatio-temporal framework guided by large language models (LLMs) to model long-term disease progression on brain graphs, utilizing structure discovery from longitudinal, population-level patient data. A key innovation lies in addressing the unknown temporal alignment of observations by embedding the method within a dual-optimization framework. This framework simultaneously estimates the disease progression model—capturing common biomarker trajectories across a population—and performs structure discovery, learning a graph that represents the spatiotemporal interactions among biomarkers. The proposed approach achieves superior prediction accuracy, faster and more stable convergence, and enhances the interpretability of the model.
1. There are a lot of typos, such as "To address these limitations, we **consder** use Large Language Models (LLMs) as
expert guides to enforce the graph inference with expert **knowlegdge**" and so on.
2. In Eq. (5), why **p**  is calculated from the **99th** percentile of the tau distribution at each region?
3. In Eq. (8), what does $D_{LLM}$ represent? Could you clarify the statement, "This not only accounts for the diffusion process along the white matter bundles but also considers additional factors related to tau accumulation from the knowledge base of various LLMs"? From Equations (7-9), I am not seeing how structural connectivity is addressed (i.e., the diffusion process along the white matter bundles).
4. In Eq. (10),  what does $X$ stand for? what's the relationship between this section and the previous sections? I can't follow this.
5. What's the dataset you use? How about the performance on other diseases? What's the sample size? How many time point for each subject do you use?
6. Insufficient experiments and lack of comparison of many disease progression modeling models.
7. This method claims to estimate regional tau-pathology propagation over the brain. Can authors provide the related propagation pattern in the brain over time?
Yes
No ethics review needed.
1. In Section A.3, why is the summation limited to four types of brain connectivities rather than five? 
2. For clarification, when comparing Model 1 and Model 2, are the authors directly comparing the performance of these two methods, or are they comparing the graphs used in these models within the context of Method 3?
1. The paper is well-written and engaging. The idea of leveraging LLM knowledge for graph initialization is particularly intriguing. Given the recent advancements in LLM, effectively incorporating this knowledge into fundamental clinical research is crucial, and the authors demonstrate this integration in a good way. 
2. The authors provide solid background information that helps readers easily grasp the context of their work. 
3. Sufficient detail is offered on critical components, including prompting strategies, clear mathematical definitions, and experimental details, allowing readers to understand the underlying processes. 
4. The evaluation of the model through critical edge numbers is both interesting and effective, showcasing the advantages of LLM-based graphs. 
5. The authors conduct an extensive analysis, covering prompt ablation, synthetic data, different contexts (both data-driven and mechanistic), and model convergence.
The authors propose a novel LLM-guided spatio-temporal framework for modeling regional variable changes during disease progression. The key innovation of this paper lies in utilizing a large language model (LLM) to enhance graph initialization and regularization in both mechanistic and data-driven settings, demonstrating improvements in accuracy and convergence. The models were evaluated on regional tau-pathology propagation across the brain.
1. I may have missed it, but the calculation of the metric "SSE" lacks a clear definition, making it challenging to assess its scale effectively. 
2. The experiments appear to be conducted on a single data split, without cross-validation or random splitting. Given the relatively small variances reported in Table 2, it’s difficult to evaluate the significance of differences and reproducibility. Incorporating additional splits and reporting standard deviations would enhance the findings. 
3. The representation of faster convergence (A.3), particularly between the second and third figures, could be clearer. Adjusting the scale might help elucidate this aspect. 
4. While the authors emphasize the importance of identifiability and interpretability, their discussion primarily revolves around reducing the number of edges in the graph. There is limited exploration or discussions on how the derived graphs enhance understanding of regional interactions and differences during disease progression compared to other graphs.
5. In figure 4, descreasing the critical edge number throgu “7-factor” prompt is not very clear to me. It seems that the other factors provided by LLM does not lead to much advantages. 
6. It appears that the model assumes a single typical progression path of phenotypes per disease, potentially overlooking disease heterogeneity. Recent research has demonstrated significant heterogeneity in disease progression, particularly in brain diseases, including amyloid and tau spreads.
@inproceedings{
anonymous2024llmguided,
title={{LLM}-guided spatio-temporal disease progression modelling},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=UTP6bdFlkB},
note={under review}
}
Understanding the interactions between biomarkers across brain regions during disease progression is essential for unraveling the mechanisms underlying neurodegenerative disease.  For example, in Alzheimer's Disease (AD) and other neurodegenerative conditions, existing models describe how variables interact with each other spatiotemporally within a dynamical system driven by an underlying causal substrate often based on brain connectivity. However, such methods typically grossly oversimplify the complex relationship between brain connectivity and brain pathology appearance and propagation. In this work, we propose a novel LLM guided spatio-temporal framework for disease progression, to model long-term disease progression on brain graphs based on discovery of structure from longitudinal population-level patient data. The new approach replaces the static brain connectome that drives most models of AD pathology propagation with a more comprehensive graph of local independencies jointly informed by human knowledge encoded in the LLM and observed patient data sets. Crucially, we formulate the method to work with typical medical data sets (cross-sectional or short-term longitudinal designs) in which the exact timepoint along a hidden underlying trajectory of change is unknown for each data point. Specifically, our method leverages a mixture of different LLMs as expert guides to constrain graph estimation. Furthermore, we account for the unknown temporal location of observations by embedding the proposed method within a dual-optimization framework, which simultaneously estimates a disease progression modelling (trajectory of change of biomarkers common to a population) and executes structure discovery (learning a graph capturing spatiotemporal influence among biomarkers). Experiments evaluate our method's performance in estimating regional tau-pathology propagation over the brain. Results demonstrate superiority over existing approaches using standard underpinning directed graphs estimated and fixed a-priori. Our method achieves higher prediction accuracy, faster and more stable convergence, and improves interpretability of the fitted model. The LLM produces additional factors that drive the disease which go beyond the factors contained in the prompt.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
disease progression modelling LLM spatio temporal modelling
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/142811303daa55cfd5fde571c1d57e5541aa779b.pdf
applications to neuroscience & cognitive science
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
LLM-guided spatio-temporal disease progression modelling
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission