Do Mice Grok? Glimpses of Hidden Progress in Sensory Cortex
neuroscience; representation learning; grokking; overtraining; cortex
Yes
No ethics review needed.
This seems like it would be a very general phenomenon, in particular because natural gradient descent dynamics should keep lowering the loss, which by construction has to maximize some discriminant in the data being classified.  At the same time, the training samples are discrete so you are guaranteed to saturate the training error (i.e. you can’t get better than perfect on a finite set).  Your kernel argument at the end seems to back up this intuition.  What properties of networks would you envision would prevent this behavior?  Or what kinds of tasks?
The paper is very clear and well presented.  The basic result is simple and backed by both empirical data and a simple model.  Most of the figures are very clear and concise.  That the observation can be explained by so simple a model is nice.  The explanation of reversal training is nice.
The authors examine the phenomenon of overtraining in posterior piriform cortex of mice.  They show that class separation continues after performance saturates.  They further show that the margin of the maximum margin classifier increases in the overtraining period.  They construct a simple model, which is a one-hidden layer MLP, that shows this overtraining behavior.  They propose that this model provides an explanation for over-training reversal in animal learning.
I think there are a few improvements that could be made to the figures:

Figure 1 could use a quantitative indication of performance, so that it is clear when overtraining starts (as opposed to this being buried in the caption).  

Figure 2 is slightly confusing in terms of which column in each row represents what.  Do I understand correctly that the first column is the average and the others are the first and last 3 days, respectively?  If so, this could be labeled better.  If not, this could be labeled much better.  

In Figure 3, maybe add an indication of the cluster mean with a line separating the clusters to make the separation more clear?  It’s not bad now, but some of these are rather close.  

In Figure 5, panel labeled “Epoch 1000”, the target dot is obscured.
Yes
No ethics review needed.
Could the authors comment specifically on:
1)	How results from the MLP model would generalize to more biologically constrained models of the PPC (or other olfactory processing circuits)?
2)	How results from MLP could generalize to more complex DL models for classification?
3)	Demonstrate a causal link between margin maximization and the emergence of grokking in the MLP model?
The study draws an interesting connection between the grokking phenomenon in deep learning and to the gradual evolution of neural representation in mice overtrained on classification tasks. By using a simple MLP model to demonstrate the link between increase in class margin and the emergence of “grokking”, the authors highlight a promising mechanism by which grokking may emerge in biological and artificial learning systems.
This study examines grokking, an interesting phenomenon observed in both deep learning models and in animal learning. The authors re-examine a published dataset that tracks neural activity in the posterior piriform cortex (PPC) of mice trained to perform a multi-odor discrimination task. The authors found that the class margin measured in the neural activity space of PPC continued to increase after performance accuracy has reached a ceiling. In a simple MLP with one hidden layer trained to perform an in silico “odor” discrimination task, the authors found continued improvement in class margin measured in latent space, and “grokking” phenomenon in probe trial accuracy. Using this simple model, the authors also provide theoretical explanation and empirical demonstration of overtrained reversal, a phenomenon long reported in experimental psychology.
1)	The architecture of the MLP model is likely far removed from the actual structure and connectivity of the PPC, hence it appears more like a toy model to demonstrate the cooccurrence of margin maximization and grokking. Whether similar phenomenon would emerge in more sophisticated models of olfactory cortex (or olfactory circuits in lower animals) remain to be tested;
2)	While the emergence of grokking in the MLP model is interesting, the authors have yet to test whether the observed increase in class margin is causally related to grokking. The authors cite related theory, e.g. setting margin maximization as an optimization objective could drive the emergence of grokking. This proposed causal link could be conveniently tested in the MLP model.  
3)	Without further analysis, it is unclear how the link between class margin maximization and grokking in more complex DL models, e.g. models trained to classify images. While the authors provide a theoretical basis for how overtrained reversal occurs, a theoretical advance on why grokking occurs remains lacking.
Yes
No ethics review needed.
Suggestions for Improvement:
1. Tone down claims about biological fidelity and instead emphasize that the model captures key computational principles at an abstract level
2. Add discussion of connections to deep learning literature on perceptual learning, which provides complementary perspectives on how representations evolve during training
3. Clarify the precise definition of margin maximization being used, particularly in the context of evolving representations.
- Makes an important connection between seemingly disparate phenomena in biological and artificial neural networks (hidden representational changes during overtraining, grokking, reversal learning)
- Provides a compelling mathematical framework (margin maximization) that helps explain the neurobiological data
- Offers a novel explanation for the classic "overtraining reversal effect" from psychology in terms of rich feature learning
- Good empirical validation through neural data analysis and synthetic modeling
- Mostly clear writing and effective visualization
This paper examines an intriguing phenomenon where neural representations in mouse sensory cortex continue to evolve and improve even after behavioral performance has reached ceiling levels (Berners-Lee et al. 2022). The authors reanalyze neural recording data from mice learning an odor discrimination task and find evidence for "hidden progress" in how odor representations separate in piriform cortex during overtraining, similar to the "grokking" phenomenon observed in deep neural networks. They propose that this continued refinement of representations reflects implicit margin maximization and demonstrate this using a simplified neural network model. The paper makes novel connections between neuroscience and machine learning while offering new insights into both domains.
- The characterization of their model as "biologically faithful" feels overstated, given the significant abstractions from real neural circuits
- The paper should cite relevant literature on deep learning approaches to perceptual learning, particularly work by Wenliang and Seitz (2019) that examines how perceptual learning can be explained as changing readouts from complex features
- The explanation of margin maximization was a bit inscrutable, particularly the part about distinguishing between different notions when a decoder is being retrained on evolving features. It could use a rewrite and tightening up, perhaps a subfigure.
Yes
No ethics review needed.
I am unclear about the contribution of the section "A mathematical model based on rich learning" on page 9. If the authors use a kernel method, isn’t the result somewhat trivial? Furthermore, the logical connection between the content of this section and the conclusion — “An animal overtrained on a task will have better task-model alignment and a richer feature set than an animal whose learning was stopped after behavioral plateau” — is unclear. Could the authors elaborate on this section?
The paper addresses interesting topics at the intersection of foundation models and cognitive science, particularly within the realm of behavioral psychology.
The paper demonstrates that animals can learn tasks and improve generalizability through over-training. The study begins by reproducing and reanalyzing experiments conducted on mice (Berners-Lee et al., 2023) and then proceeds to run toy experiments as proof-of-concept illustrations.
**1. Writing Quality:**

* *Citation:* The poor writing significantly hinders the evaluation of this work. For instance, the authors misuse \citet and \citep throughout the paper. The frequency of these errors affects the paper’s professionalism; please revise these accordingly.

* *Confusing Notation:*
Although Berners-Lee et al. (2023) represent each odor as a vector in their figures, the description "n-hot vector of $k$ possible odorants" may confuse readers. Given that ICLR’s primary audience is in machine learning, they may interpret this as a traditional vector representation, though in the authors' experiment it represents a combination of odorants rather than a vector.

* *Undefined Notations:*
Several notations lack definitions (e.g., [N, T] in line 755) or exact values (e.g., $n$ in $n$-hot vector).

**2. Validity of Experimental Setting:**

The results could be heavily dependent on the experimental setting. In the default setup, each odor is represented as a combination of odorants, defined by an $n$-hot vector, with limited overlap between odorants. Given the network’s simple architecture (a single fully connected layer), each odorant is almost linearly separable, meaning there is minimal interference among components. This could explain why the model resists overfitting and maintains increasing test accuracy even during overtraining. Conversely, as seen in Figure 6, when odor overlap is greater, neither the Fisher Linear Discriminant nor the test accuracy increases significantly beyond 100 epochs, at which point the model reaches perfect training accuracy.

**3. Loss Interpretation:**

The authors claim in line 370 that after achieving perfect training accuracy, the loss "plateaus." However, it appears to continue decreasing (though this could be subjective). Additionally, perfect training accuracy does not imply the model has fully learned from the training data, especially since cross-entropy loss is used, and accuracy is based on the argmax classification.

**4. Miscellaneous**

* Lines 457–484: This section needs refinement. It is unclear how the original features differ from pre-trained features (L461–463), the relationship between $K$ and $K_0$ (L465), and the contrast between the original model’s readout and the randomly initialized network (L482).

* Figure 1: The caption refers to (a) and (b), but these labels are absent in the figure.

* Line 355: “are are” should be corrected to “are.”
We find evidence for rich feature learning in mouse piriform cortex during overtraining and propose it is driven by approximate margin-maximization, a known cause of grokking in deep learning.
@inproceedings{
anonymous2024do,
title={Do Mice Grok? Glimpses of Hidden Progress in Sensory Cortex},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=oYemKnlIrO},
note={under review}
}
Does learning of task-relevant representations stop when behavior stops changing? Motivated by recent work in machine learning and the intuitive observation that human experts continue to learn after mastery, we hypothesize that task-specific representation learning in cortex can continue, even when behavior saturates. In a novel reanalysis of recently published neural data, we find evidence for such learning in posterior piriform cortex of mice following continued training on a task, long after behavior saturates at near-ceiling performance ("overtraining"). We demonstrate that class representations in cortex continue to separate during overtraining, so that examples that were incorrectly classified at the beginning of overtraining can abruptly be correctly classified later on, despite no changes in behavior during that time. We hypothesize this hidden learning takes the form of approximate margin maximization; we validate this and other predictions in the neural data, as well as build and interpret a simple synthetic model that recapitulates these phenomena. We conclude by demonstrating how this model of late-time feature learning implies an explanation for the empirical puzzle of overtraining reversal in animal learning, where task-specific representations are more robust to particular task changes because the learned features can be reused.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
neuroscience; representation learning; grokking; overtraining; cortex
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7e3e184e5f266bb41d30252763c732af68548485.pdf
applications to neuroscience & cognitive science
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Do Mice Grok? Glimpses of Hidden Progress in Sensory Cortex
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission