Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization
Language models;simplicity bias;random variations;OOD generalization
Yes
No ethics review needed.
Questions
===
1. Where did the hyperparameters in L212-215 come from? Was there some tuning involved, and if so, would it be possible to show results for other tested hyperparameters?
2. By syntactic complexity, do you generally mean syntactic tree depth? The rarity/difficulty for humans in processing of particular structures? Number of edges in the tree (which could correlate with sentence length, even if the maximum depth of the tree were low)? I assume the first given the analyses in Sec. 4, but would be nice to explicitly define the term.
3. L420: which studies?
4. At first glance, Fig. 4 and Sec. 6 feel contradictory. There is some interesting nuance here that could be interesting to explore: diversity is necessary to prevent memorization, but at the same time, including a diverse mixture of complex and simple sentences leads to *worse* generalization than simply using more complex sentences. Why is this?

Suggestions/Typos
===
* L182: there are *at least* two strategies. The model could rely on other heuristics, such as always moving the affirmative verb in QF as opposed to the main or first verb (as attested in Mueller et al., 2024).
* Table 1 caption: by “left” and “right”, do you mean “top” and “bottom”?
* L113: two periods
* The paper adds to a rich literature on directly evaluating the inductive biases of language models. This paper approaches it from a grokking perspective, which, to my knowledge, has not been done before.
* In Fig. 4, it is interesting that such a small variety of syntactic structures in the declaratives would lead to hierarchical generalization. I had assumed that a more naturalistic and varied distribution would be necessary to prevent memorization.
* Thorough analysis of variation across many random seeds.
This study analyzes the role of the syntactic complexity in language models’ acquisition of hierarchical inductive biases. Specifically, by generating sentences with varying syntax tree depths and of varying structures (some with mixed scoping and others with forward scoping), the features that lead to linear or hierarchical generalization are isolated. Additionally, by mixing these types of data, the authors conduct a grokking-style analysis of when stable, unstable, hierarchical, linear, or mixed generalizations are likely to be learned. 

It is found that including sentences with a syntactic tree depth of at least 3 is required for hierarchical generalizations to be stably acquired, and that mixing depths in the training data can lead to unstable generalizations. It is also found that if the data is not sufficiently diverse, memorization (rather than any consistent generalization) is the preferred strategy.
1. The findings could be much better contextualized with related work throughout the paper. For example, at L150: this is not the first work to analyze the effect of data on syntactic generalization. Mueller et al. (2023) (who are currently cited only in the Appendix) do a similar analysis, finding that simpler corpora lead to syntactic generalization with less data. Additionally, when referring to the syntactic transformations task setup at L144, Frank & Mathis (2007) [1] is the correct citation. At L154-161 (and App. F): it would be nice to contextualize this with analyses from McCoy et al. (2020) on instability across random seeds. In Sec. 4.3, these findings are very related to those of Papadimitriou & Jurafsky (2023) [2] (who are not cited) on training with recursive and/or cross-serial dependencies.
2. The findings are somewhat obvious in light of the above citations, and in light of grokking work (which, by the way, should also be better cited and discussed; see [3,4,5]). Data determines generalization because neural networks are a statistical approach to learning. There are probably new insights to be gleaned from this new task setting compared to past grokking work, and it could be nice to explicitly enumerate these in the paper.
3. L212-215: It is not clear why these hyperparameters were chosen, and why different hyperparameters were used for the two tasks.
4. Related to the first point (but more minor), the decision of what to include in the main paper vs. the appendix Related Work sections currently feels arbitrary.
Yes
No ethics review needed.
- It might be worthy study how other objectives (like MLM) interact with data composition to affect rule learning. 
- Have you considered how the role of data composition might vary with languages that have different syntactic structures?
- Interesting analysis of the mechanisms of rule commitment.
- The identification of a memorization regime, where models stabilize without learning either hierarchical or linear rules.
- The findings are replicated across two distinct grammatical tasks, and backed up by linguistics theroies.
This paper investigates the influence of training data composition on the generalization behaviors of language models, focusing on their ability to learn hierarchical syntactic representations versus relying on surface-level heuristics such as n-gram models. Through case studies involving English grammar tasks (specifically question formation and tense inflection) the authors show that the complexity and diversity of training data play pivotal roles in determining if models adopt hierarchical rules or simpler linear heuristics. 
Interesting findings of the paper: 
i) Adding sentences with deep syntactic trees in the training data encourages models to develop hierarchical syntactic representations  (and this seems to enable out of distribution generalization).
ii) When the training dataset have a mix of simple and complex grammatical structures, models show unstable training dynamics and inconsistent rule commitments across different random seeds (this aligns with findings by McCoy et al. (2018, 2020))
iii) In case of low data diversity, models tend to memorize patterns without learning robust hierarchical or linear rules, resulting in poor generalization.

I think the main message of this paper is the relevance of training data features in shaping the inductive biases of neural networks.
- The experiments utilize relatively small transformer models trained on synthetic datasets with 100K samples (it doesn't make the study less valid, but larger models may exhibit different inductive biases and learning dynamics that are not captured by smaller-scale experiments).
- The authors use a fixed set of hyperparameters (learning rate of 1e-4, Adam optimizer, specific layer configurations) across all experiments, without addressing how variations in hyperparameters might influence the model's ability to learn hierarchical rules or affect training stability.
Yes
No ethics review needed.
See in Weaknesses.
The paper is well-written, with comprehensive visuals and thorough experimental results.
This paper investigate generalization of LMs.  Specilically, data composition significantly influences a model's out-of-distribution (OOD) generalization.  Insufficient data diversity can lead models to rely on memorization rather than achieving true generalization. They primarily validate these observations through learning on grammatical tasks.

model: 12M  decoder-only Transformer.
tasks: question formation, tense inflection.
1. The conclusion in this paper have many overlaps with previous studies[1],[2],[3]; please specifiy your difference and contribution detailed.
2. The experiments only contain two dataset: question formation and tense inflection. This limited validation is insufficient to support the overall conclusions of the paper, generally speaking.  More tasks and models are needed.  Suggest tasks: language modeling, mathematics reasoning … …
3. The authors present only the experimental observation that "lower diversity data tends to promote memorization, while higher diversity data encourages generalization." These are  obvious conclusions compared with previous papers. Are there any new insights offering deeper theoretical reasoning or more controlled experiments to support these findings? or fuerther verfiication on LLMs, e.g. 7B ?

### Grammar

Line 113: “. . “

Line 119: multiple abbreviation definitions

Table 1: revise “Left” and “right” to “Top” and “Bottom”

[1] Liu Z, Michaud E J, Tegmark M. Omnigrok: Grokking beyond algorithmic data[C]//The Eleventh International Conference on Learning Representations. 2022.

[2] Zhu X, Fu Y, Zhou B, et al. Critical data size of language models from a grokking perspective[J]. arXiv preprint arXiv:2401.10463, 2024.

[3] Wang B, Yue X, Su Y, et al. Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization[J]. arXiv preprint arXiv:2405.15071, 2024.
Data shapes language model's ability to overcome simple heuristics and achieve complex generalization behaviors.
@inproceedings{
anonymous2024sometimes,
title={Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=juxbsQEuTZ},
note={under review}
}
Neural networks often favor shortcut heuristics based on surface-level patterns. As one example, language models (LMs) can behave like n-gram models early in training. However, to correctly apply grammatical rules, LMs must instead rely on hierarchical syntactic representations rather than the surface-level heuristics derived from n-grams. In this work, we use cases studies of English grammar to explore how latent structures in training data can enable models to overcome their early capabilities and achieve complex generalization behaviors. We demonstrate that sentences with complex grammatical structure drive the model's inductive bias towards hierarchical representation. We then investigate how data composition can lead to inconsistent behavior across random seeds, finding that models stabilize in their out-of-distribution (OOD) behavior only when they commit to either a surface-level heuristic or a hierarchical rule. When the data contains a mix of simple and complex examples, potential rules compete, leading to unstable dynamics in training runs that fail to commit. We also identify an exception to the relationship between stability and generalization: Models which memorize patterns from homogeneous training data can stabilize in a memorization regime without learning either rule. While existing works have attributed similar generalization behavior to training objective and model architecture, our findings emphasize the critical role of training data in shaping generalization patterns and how competition between data subsets contributes to inconsistent generalization outcomes.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Language models simplicity bias random variations OOD generalization
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/511cc281a7f805e6eda82e803a5be60026bf86f7.pdf
interpretability and explainable AI
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/695e62b1ecdc650c2c339951060ec53ecfc1e768.zip
Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission