Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment
long-tailed recognition;imbalanced learning;neural collapse;logit adjustment;multiplicative logit adjustment;machine learning;learning theory
Yes
No ethics review needed.
Please refer to the weakness section.
- This paper presents a theory for optimally adjusting decision boundaries based on NC and links it to the MLA method, providing a strong theoretical explanation for MLA and adding depth to its empirical success.
- The authors conduct a series of experiments on long-tailed datasets to validate the theory and demonstrate the practical utility of MLA.
- The authors offer both theoretical and empirical comparisons between MLA and ALA, clarifying their differences and highlighting the advantages of MLA.
This paper provides a theoretical foundation for the Multiplicative Logit Adjustment (MLA) method used in long-tailed recognition tasks. First, the authors develop a theory for optimally adjusting decision boundaries by leveraging feature spread estimates derived from neural collapse (NC). They then demonstrate that MLA effectively approximates this optimal adjustment method. Experiments conducted on various long-tailed datasets validate the practical applicability of this approximation, showing that MLA performs well under realistic conditions.
- The iNaturalist dataset is a widely used large-scale long-tailed dataset. Conducting experiments on this dataset would be beneficial to examine the behavior of MLA, ALA, and the 1vs1 adjuster in a larger-scale real-world context. Additionally, testing on CIFAR-LT with more extreme imbalance ratios would further evaluate the effectiveness of the proposed method.

- The theoretical framework relies on some strict assumptions that may not always hold in practice, such as the assumption of conditions under which NC occurs, which may not be valid in highly imbalanced real-world datasets. Similarly, the assumption that $\psi \rightarrow \pi/2$ does not hold when $K$ is small.
- There are some typos, such as on line 373, where "the optimal decision boundaries during inference is effected by" should be corrected to "the optimal decision boundaries during inference are affected by".
Yes
No ethics review needed.
The paper mentions that "MLA has demonstrated significant empirical success," but it does not provide sufficient evidence to support this claim. Could more literature or experimental results be added to demonstrate the current advantages of MLA in long-tailed recognition?

The theoretical comparison between MLA and ALA is addressed in some aspects, but why is there no comparison with other advanced baselines in long-tailed recognition (such as the latest contrastive learning methods or distribution-balanced loss)?
The framework that connects MLA with Neural Collapse is novel.  The assumptions and conditions are clearly stated, and the notation is consistent and well-defined.
The paper aims to provide a theoretical justification for Multiplicative Logit Adjustment (MLA) in long-tailed recognition by linking it to Neural Collapse theory. The authors develop a theoretical framework for optimal decision boundary adjustment based on feature spread estimates from Neural Collapse and demonstrate that MLA approximates this optimal method.
1. **Motivation**:
   The paper fails to establish a compelling motivation for the study. The claim that "MLA has demonstrated significant empirical success" is not sufficiently substantiated. There is no systematic analysis of the current strengths and challenges of MLA in long-tailed recognition.

2. **Theory-Practice Gap**:
   The connection between optimal decision boundary adjustment and actual performance improvements in long-tailed scenarios is weak. While the paper seems to demonstrate the effectiveness of MLA as a close approximation to the 1vs1adjuster from both theoretical and experimental perspectives, and contrasts it with Additive Logit Adjustment (ALA), it does not thoroughly analyze how MLA addresses the specific challenges of long-tailed learning from a theoretical standpoint. Additionally, there is no comparison with state-of-the-art long-tailed baselines or discussion on how MLA could be integrated with them. The experimental section focuses more on hyperparameter tuning for MLA rather than examining its impact on long-tailed learning.
Yes
No ethics review needed.
1. Why did you primarily focus on ResNet models for the image analysis? Were there specific reasons for this choice over other architectures?
2. Why did you create imbalanced versions of balanced datasets instead of using inherently imbalanced datasets? For example, datasets that exhibit natural class imbalances could offer more realistic evaluations.
3. Although the paper focuses on post-hoc LA methods, did you compare the results with loss-function-based approaches? How does MLA's performance compare to other logit adjustment methods?
* The problem of class imbalance is clearly motivated, and the investigation into MLA is a valuable contribution to this area of research.
* The mathematical framework is well-developed, offering a solid theoretical foundation for understanding the method.
* The analysis extends beyond image datasets to include other modalities, such as text, enhancing the paper's applicability.
* Extensive experiments are conducted, which reinforce the theoretical claims and provide a comprehensive evaluation of MLA’s performance.
The paper addresses the problem of class imbalance in classification models and analyzes the effectiveness of the Multiplicative Logit Adjustment (MLA) method. The authors propose a theory on optimal decision boundaries based on neural collapse, demonstrating that MLA approximates these optimal boundaries. The paper is supported by experiments across various modalities, providing insights into MLA’s performance under different conditions.
* While the theoretical concepts are accurately communicated, the writing could be improved by breaking up overly long sentences to enhance readability (e.g., lines 098-104).
* The results in Tables 6-10 are intriguing, but more explanation is needed to clarify the implications of "Many," "Medium," and "Few" categories. Additionally, the reasons for the Baseline outperforming others in the "Many" category should be discussed in more depth.
* The descriptions for tables and figures are too brief, making it difficult for readers to grasp the full context of the results. Adding more detailed captions would help in understanding the findings.
* The paper contains numerous formal definitions and formulas, which can disrupt the reading flow, especially for readers who are not deeply familiar with this specific research area.
We establish a theoretical basis for understanding the effectiveness of multiplicative logit adjustment in long-tailed recognition.
@inproceedings{
anonymous2024multiplicative,
title={Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=II81zQUS1x},
note={under review}
}
Real-world data distributions are often highly skewed. This has spurred a growing body of research on long-tailed recognition, aimed at addressing the imbalance in training classification models. Among the methods studied, multiplicative logit adjustment (MLA) stands out as a simple and effective method. What theoretical foundation explains the effectiveness of this heuristic method?
We provide a justification for the effectiveness of MLA with the following two-step process. First, we develop a theory that adjusts optimal decision boundaries by estimating feature spread on the basis of neural collapse. Second, we demonstrate that MLA approximates this optimal method. Additionally, through experiments on long-tailed datasets, we illustrate the practical usefulness of MLA under more realistic conditions. We also offer experimental insights to guide the tuning of MLA hyperparameters.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
long-tailed recognition imbalanced learning neural collapse logit adjustment multiplicative logit adjustment machine learning learning theory
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/5447df91b48f08d0a1e1b6bf9d7d5c89b93d7357.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission