UNICORNN: Unimodal Calibrated Ordinal Regression Neural Network
unimodality;ordinal regression;probability calibration;deep learning
Yes
No ethics review needed.
1. Please comment on the rank consistency of the proposed approach. In think, unimodularity of the output distribution and rank consistency of the two contradictory properties. Please comment. It would be good to see the average number of rank violations made by the proposed approach.
2. Please compare with a rank-consistent baseline (e.g. Shi et al., 2023)
3. A simple approach for imposing unimodularity in the output vector is by posing the constraint $P(Y=1)\leq P(Y=2)\leq \ldots \leq P(Y=c)\leq \ldots P(Y=K)$, where $c$ be the target class and $K$ be the total number of classes. Minimizing loss with these constraints can be an alternate approach to achieving unimodularity. It can be a good baseline for the proposed approach. Can you show some results why such an is not good?
4. Please explain the truncated normal distribution properly. What is the support set for this distribution?
1. Authors propose a theoretically sound approach for ordinal regression, which can output unimodular outputs.
2. Paper is very well written and easy to read.
In this paper, the authors propose an ordinal regression approach that enforces unimodularity in the output distribution. They use a truncated normal distribution to model the output probabilities.  They provide unimodular proof of the output distribution. They show experimental results to highlight the effectiveness of the proposed approach compared with baseline approaches on various metrics.
1. Some references are missing. E.g.
(a) Rank consistent ordinal regression for neural networks with application to age estimation. Wenzhi Caoa, Vahid Mirjalilib, Sebastian Raschkaa. Pattern Recognition Letters, (2020), 325-331.
(b) Garg, B. &amp; Manwani, N.. (2020). Robust Deep Ordinal Regression under Label Noise. Proceedings of The 12th Asian Conference on Machine Learning. 129:782-796

2. The baseline approaches used do not cover a variety of algorithms for ordinal regression. Please see the Questions section for suggested baselines.

3. Code is not provided.
Yes
No ethics review needed.
**Questions**:

Please address the weaknesses mentioned above.
**Strengths**:

1. The paper clearly describes the motivations and explanations of the proposed model.
2. A rather simple and interesting approach for ensuring unimodality of the predictive distribution.
3. Retraining procedure for producing calibrated distributions.
4. The experiments show the benefit of the proposed approach on several datasets.
**Summary**:

The paper presents a technique for ordinal regression tasks, focusing on two aspects: (1) constructing uni-modal distribution, which is desirable in many cases; (2) generating calibrated predictive distribution that is comparable to the empirical distribution. For the uni-modal distribution construction, it generates a truncated normal distribution, and constructs evenly spaced thresholds to ensure the unimodality. For the calibration, the model performs retraining using Brier loss to adjust the parameter from the first stage. The authors then demonstrate the performance of the proposed model in the real experiments.
**Weaknesses**:

1. The description of the proposed training algorithm in Sec. 4.4 is rather simple. I would suggest the authors to put a proper algorithm (using formal notation) for the model
2. The way the authors ensure unimodal distribution can be limiting. Even though the proposed method always generates a unimodal distribution, not every unimodal distribution can be represented by the proposed construction. In fact, with just two degrees of freedom, it is rather limited.
3. The novelty of the proposed models is rather limited, as similar unimodality construction and the calibrated distributions constructions have been proposed before.
Yes
None
No ethics review needed.
See weakness
1. UNICORNN addresses significant limitations in existing ordinal regression methods by providing an approach that ensures both unimodal and calibrated probabilities, which is critical for applications 
2. The paper is well-structured and Explanations are generally clear.
3. The combination of Optimal Transport (OT) loss with accuracy-preserving calibration is original and significant for ensuring both order respect and reliable probability estimates
This paper introduces UNICORNN, a deep-learning approach designed for ordinal regression tasks where class labels follow a natural order. The authors address three primary challenges in ordinal regression: (1) enforcing unimodal output probability distributions, (2) capturing the ordered relationships among classes effectively, and (3) providing well-calibrated probability estimates. Extensive experiments on six real-world datasets demonstrate that UNICORNN consistently matches or outperforms recent ordinal regression methods, excelling in accuracy, probability calibration, and ensuring output unimodality.
1. The proposed method has a two-stage training process, significantly increasing the model's computational complexity. Can the author show the time complexity of training or compare the training time of the proposed method with other methods?
2. The methods used in experiments are too old. The authors should include comparisons with methods from recent years such as [1,2]. Can the author show the results on some large-scale datasets, such as the datasets with more training samples or more classes? 
3. More experiments are needed in ABLATION STUDY. It only has two datasets. More experiments on different datasets are required.  


[1] Shi X, Cao W, Raschka S. Deep neural networks for rank-consistent ordinal regression based on conditional probabilities[J]. Pattern Analysis and Applications, 2023, 26(3): 941-955.
[2] Li Q, Wang J, Yao Z, et al. Unimodal-concentrated loss: Fully adaptive label distribution learning for ordinal regression[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 20513-20522.
[3] Kim D, Chung H, Jang I. Calibration of ordinal regression networks[J]. arXiv preprint arXiv:2410.15658, 2024.
A deep ordinal regression model for unimodal and calibrated output probabilities.
@inproceedings{
anonymous2024unicornn,
title={{UNICORNN}: Unimodal Calibrated Ordinal Regression Neural Network},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=dMj3SDNxn4},
note={under review}
}
Ordinal regression is a supervised machine learning technique aimed at predicting the value of a discrete dependent variable with an ordered set of possible outcomes. Many of the algorithms that have been developed to address this issue rely on maximum likelihood for training. However, the standard maximum likelihood approach often fails to adequately capture the inherent order of classes, even though it tends to produce well-calibrated probabilities. Alternatively, some methods use Optimal Transport (OT) divergence as their training objective. Unlike maximum likelihood, OT accounts for the ordering of classes; however, in this manuscript, we show that it doesn't always yield well-calibrated probabilities. To overcome these limitations, we introduce UNICORNN, an approach inspired by the well-known Proportional Odds Model, which offers three key guarantees: (i) it ensures unimodal output probabilities, a valuable feature for many real-world applications;
(ii) it employs OT loss during training to accurately capture the natural order of classes;
(iii) it provides well-calibrated probability estimates through a post-training accuracy-preserving calibration step.
Experimental results on six real-world datasets 
demonstrate that UNICORNN consistently either outperforms or performs as well as recently proposed deep learning approaches for ordinal regression. It excels in both accuracy and probability calibration, while also guaranteeing output unimodality. The code will be publicly available upon acceptance.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
unimodality ordinal regression probability calibration deep learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ee03d5f5565034a11ee8c3cf7a28ed9577bf9831.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
UNICORNN: Unimodal Calibrated Ordinal Regression Neural Network
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission