Toward a Sheaf-Theoretic Understanding of Compositionality in Large Language Models
cognition;compositionality;sheaf-theory;language model evaluation
Yes
No ethics review needed.
N/A
- The paper presents a nice initial effort in defining compositionality with a mathematical framework rigorously
- The paper is well-structured and well-written
The paper introduces a mathematical framework for compositionality in LLMs based on sheaf topology, defining four basic conditions: restriction maps, gluing conditions, locality conditions, and natural transformations. The authors tested each condition with a specific dataset and found that instruction-tuned models have inconsistent performance across different aspects of compositionality.
- (Major) Limited novelty and applicability of the findings:
	- The formalization of compositionality via the sheaf-theoric framework is novel, but it is not clear its purpose, how it can be used in practice in the real world and whether it's correct or not (due to a weak experimental part).
	- There is no novelty in the evaluation part since it uses known datasets and results for the literature, except for the introduction of the COMPCOMB dataset.
	- In general, there are very few contributions that justify this paper.
- (Critical) The experimental part is extremely weak and not convincing:
	- In Table 1 there is a significant drop between base models and instruction-tuned models like a huge ~40% (0.82 vs 0.42) on SCAN. The authors provide an explanation that  L402 "instruction tuning likely leads to a loss in the development of restriction maps, which could be explained by the fact that while the model retains its most important generalizations, it loses some local information to accommodate instruction tuning, leading to loss of restriction mapping". This is in contrast with results in the literature where instruction-tuned models perform generally better. I believe there is not enough empirical evidence to sustain this statement and the huge performance drop might be due to issues in the evaluation setup. The authors mention the use of "computing the model’s log probabilities for two possible completions" but this has been shown to be problematic, especially in instruction-tuned models that might lose the calibration in their logits after RLHF (https://arxiv.org/abs/2402.14499 , https://arxiv.org/abs/2303.08774). I suggest using a different eval strategy (e.g., comparison with the ground truth and exact match metric) and distinguishing results between instruction-tuning and models tuned via RLHF.
	- In general, I don't think the experiments are thorough enough to convince me without any doubt that an increase/decrease of the score on a specific dataset (e.g., SCAN) means an increase/decrease of a compositional condition defined in the framework (e.g., Restriction Condition). There are several other factors involved in the evaluation that might lead to spurious correlations and an increase/decrease in the score. I think the author should definitively pay more attention to the evaluation of the components defined in the framework.
- (Minor) The presentation of the results is not optimal:
	- The results proposed (Figures and Tables) are never referenced from the text. This creates ambiguity in the text because it's not clear what results you are commenting on. 
	- The proposed plots lack clarity. The rationale for using a radar plot to compare accuracies in Figure 1 is unclear, and both the scale and raw values in the plot are difficult to interpret.
- (Minor) There is a minimal discussion of related works. Missing related papers (e.g., [A Complexity-Based Theory of Compositionality](https://arxiv.org/abs/2410.14817))
Yes
No ethics review needed.
- Is llama2 chat hf an instruction-following checkpoint?
- Would it be possible to include Qwen in the evaluation for additional trends and patterns revealing? Also llama released 3 and 3.1.
- When introducing 4 tasks, the ordering is not consistent, is there any special consideration? 
For example, the order in table 1 or figure 1 (1SCAN, 2ANTAILS, 3COMPCOMB, 4PLANE) is different from the order in section 2 and section 3.3. 
- What is the size of each dataset?
- Could you provide an example for each dataset? 
- Could chain-of-thought prompting apply during evaluations?
- Paper investigated both query (aka, prompt output) and internal representation in different tasks.
- To reviewer's knowledge, this paper is clear and original in proposing the sheaf-theoretic framework for LLMs compositionality assessment.
- Two orders of relationship are investigated, entity level and relation level.
- Proposed 4 datasets, namely SCAN, ANTAILS, COMPCOMB, PLANE, aim to unveil insights in restriction maps, gluing condition, locality condition, and natural transformation. 
- Appendix provided abundant information about generation process of each dataset.
Compositionality is a topic on human cognition. 
LLMs appear to show their superior language processing capability and thus evaluating the compositionality become a portal to gain insights into these models.
This paper introduced a sheaf-theoretic framework with 4 different datasets to assess LLMs performance on compositionality.
Key findings include 1) larger models tend to be more performant. 2) instruction finetuned models may behave inconsistently in different tasks.
- A related work section would help readers to better understand the background and prepare readers well to follow the sheaf-theoretic framework.
- While 4 tasks cover distinct aspects, the size of these dataset could be limited to capture the compositionality of LLMs.
- Compositionality is an important in human cognition and investigating it in LLMs is also exciting. The paper would be more complete if authors include the importance of LLMs compositionality in applications. What are the aspects or benefits if LLMs gains better compositionality performance.
- There is limited description of the experiment setup. It would be preferable if there are more justifications for the methodology and choice of prompting.
- (line 908) There is a table or figure missing about the SCAN setup, showing **??**
- Appendix also shows the template used but it would be better to include additional examples in each dataset to help readers gain insights.
- There are many extremely long sentences, which requires a second pass and thus affect the readability of the paper.
Yes
No ethics review needed.
1. Are there any situations or specific tasks where the proposed sheaf-theoretic framework might not be applicable?

2. The performance shifts observed in Table 1 indicate differences across various models. What insights can these performance variations provide regarding model architecture, training methodologies, or the training data used?
1. The paper introduces a completely new, higher-level definition of compositionality using sheaf theory, which provides a fresh perspective on evaluating the compositional abilities of LLMs. This novel approach broadens the understanding of how complex linguistic expressions are structured and processed by LLMs. Provide a more comprehensive way to measure LLMs' ability.

2. The authors evaluate various LLMs (such as Llama2, CodeLlama, and Mistral) across four different angles, providing a detailed comparison of their compositional performance. This analysis helps identify which models excel in certain tasks and where they fall short, offering useful insights for improving future models.
1. This work proposes a novel way to use sheaf theory for compositionality for Large Language Models (LLMs). Compositionality is a central concept of human cognition that understands complex things through simpler components. This new definition of compositionally contains four distinct conditions: restriction maps, gluing conditions, locality conditions, and natural transformations.

2. The experiments conducted across multiple LLMs (such as Llama2, CodeLlama, and Mistral) show that larger models tend to exhibit better compositional abilities overall. However, instruction-tuned models experience a significant decline in performance, particularly in tasks related to the restriction condition and natural transformations. This suggests that while instruction tuning may enhance generalization, it can degrade the model's ability to handle compositional information.
1. The paper does not include a dedicated Related Work section, which is critical for situating the proposed framework within the existing literature. This omission makes it difficult to understand how the new approach builds upon it.

2. The paper falls short of the 10-page limit. This space can be used to conduct analysis deeper.

3. While the proposed sheaf-theoretic framework is innovative and offers a high-level definition of compositionality, its real-world applicability remains uncertain. The framework may be too abstract or theoretical for immediate use in practical model development or evaluation, raising questions about its tangible impact.

4. The paper does not provide a better solution for improving compositionality in LLMs.
This paper uses a sheaf-theoretic approach to understand and evaluate compositionality in LLMs.
@inproceedings{
anonymous2024toward,
title={Toward a Sheaf-Theoretic Understanding of Compositionality in Large Language Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=srOVvTzgPo},
note={under review}
}
Compositionality has long been considered a central component of human cognition, enabling us to adeptly learn, manipulate and generate natural language. But what does this concept mean for Large Language Models (LLMs) that strive to imitate our linguistic knowledge? How can we evaluate it in LLMs? In this study, we introduce a novel sheaf-theoretic framework to redefine compositionality for LLMs, outlining four distinct conditions that encapsulate this complex phenomenon. Additionally, we propose targeted tasks to assess these conditions and evaluate them in a diverse array of LLMs. Our preliminary findings offer insights into the behaviour of LLMs, potentially identifying causal factors that influence their overall performance trends. This research emphasizes the potential utility of our sheaf-theoretic framework for compositionality -- both as a tool for understanding LLM  behaviour and for providing a set of metrics for analysing model performance and evaluation.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
cognition compositionality sheaf-theory language model evaluation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/368076259ecc5d83442cd0cc9e97502eec36c6ae.pdf
applications to neuroscience & cognitive science
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Toward a Sheaf-Theoretic Understanding of Compositionality in Large Language Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission