Reformulating Strict Monotonic Probabilities with a Generative Cost Model
monotonic model;variational inference;generative model
Yes
-
No ethics review needed.
- How wide are the confidence intervals for each of the metrics? Just do standard bootstrap if possible and report please.
- Why introduce the extra prior $p_\theta(Z)?$ Also, why is the prior different from $\pi(Z)$?
- Clean reformulation of the monotonic problem into a classification over a latent variable.
- Well-motivated techniques to solve the problem.
- Comparison against a variety of baselines.
The paper models a conditional monotonic distribution as a marginalization over latent variables. The key idea is to modify the monotonic modeling problem into modeling an element-wise cumulative distribution function (over a cost variable C). To actually model the latter, the authors introduce a latent variable modeling problem and solve it via a standard importance-weighted likelihood estimate with or without an additional ELBO term. The paper presents improved results in two experiments over a variety of baselines.
No confidence intervals in the results tables. It's not clear that the improvement achieved by GCM is large enough (even over the simple MLP) to justify the much more complicated modeling procedure.
Yes
No ethics review needed.
No additional questions.
The paper tackles an interesting problem of modeling monotonic probabilities. The problem itself is important, and the reformulation proposed by the paper is unique. Extensive experiments are conducted to support this new method.
This paper proposes a novel approach to model strict monotonic probabilities. Without loss of generality, it considers the binary classification formulation $y|x, r\sim Bernoulli(y;G(x,r))$, where $G(x,r)$ is a function that is monotonic in $r$. The target is to learn the function $G$. Instead of directly learning $G$, the paper introduces a cost variable $c$ and reformulates $G(x,r)$ as an integration $\int_{c<r} p(c|x)d c$. The paper then introduces a generative cost model to approximate the conditional distribution $p(c|x)$.
The paper lacks theoretical results on the finite sample efficiency of the proposed algorithm. For the experiment design, an important setup that requires strict monotonicity is quantile regression, where the conditional quantile should be monotonic in the quantile argument. It would be interesting to see experiments designed for it and a comparison with the existing benchmarks.
Yes
No ethics review needed.
1. Can the authors provide additional justification or empirical validation for the assumption of conditional independence between the latent variable z and revenue r given x, as this assumption may affect model flexibility in real-world applications?

2. Could the authors elaborate on the computational efficiency of the Generative Cost Model (GCM) compared to traditional monotonic models, especially when applied to high-dimensional datasets? How does the computational time compare to the benchmarks?
The paper seems to present a novel approach to formulate the problem of strict monotonic probability, it provides convincing theoretical analysis and emprical results.
The paper proposes the Generative Cost Model (GCM) to enforce strict monotonicity by modeling a latent cost variable, using variational inference and importance sampling. This approach avoids architectural constraints and outperforms traditional methods on synthetic and real datasets.
The paper lacks empirical validation across diverse, high-dimensional real-world datasets, limiting the demonstrated generalizability of the Generative Cost Model (GCM). Moreover, the code was not provided in the supplementary materials to assess the reproducibility of the results.
Yes
No ethics review needed.
Overview with some questions embedded:

Lines 054-059 could be clearer with a graphical model diagram. Edit: there is one later, nice.

Line 063 may not be quite right; p_theta(x,r) cannot be ignored in the evidence unless you drop the dependence on theta. Maybe rework the setup / explanation a bit, this shouldn’t be a big issue.

Line 101 equivalent to what?

Line 150 consider the machinery within https://arxiv.org/abs/2301.11695 and also the separate line of work on normalizing flows that all involve e.g. invertibility, monotone transformations, etc.

Line 210 (0,1) should be {0,1}

Line 189 should **y** and r have the same dimension?

Line 289 elementwise

Line 304 looks like it would suffer from high variance since pi(z) is fixed but p(z|x) depends on x.

Line 323 the combination of losses looks a little suspicious.

Line 323 Note that IWAE (Burda) is equal to ELBO (Jordan et al) for IWAE number of samples set to one; why is adding ELBO to IWAE sensible? Shouldn’t pi be something else and then no ELBO? Please expand, this is unconvincing.

Line 324 it seems like this doesn’t really match the beta vae setup; it would if you put a beta in front of the kl in line 318. What is this doing?

Line 378 how about some ablations of alpha and beta terms? Edit: there are some in appendix C.

Table 1 does not seem to match appendix C for any value of the parameters, what is missing? Please add details.

Line >= 378 how did you set all of the parameters?

Line 698 affect
It is an interesting problem and their construction is quite novel and creative.
The paper proposes a latent variable model capturing both a monotonic part and a part that need not be monotonic. They give some background, and then propose a loss function for a variational training with neural nets.
The objective function and implementation leaves a lot of details missing and unexplained. The paper does not inspire confidence in the method.
We propose a generative method that solves the monotonic modeling task.
@inproceedings{
anonymous2024reformulating,
title={Reformulating Strict Monotonic Probabilities with a Generative Cost Model},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Qohoh5x1Jm},
note={under review}
}
In numerous machine learning contexts, the relationship between input variables and predicted outputs is not only statistically significant but also strictly monotonic. Conventional approaches to ensuring monotonicity primarily focus on construction or regularization methods. This paper establishes that the problem of strict monotonic probability can be interpreted as a comparison between an observable revenue variable and a latent cost variable. This insight allows us to reformulate the original monotonicity challenge into modeling the latent cost variable and estimating its distribution. To address this issue, we introduce a generative model for the latent cost variable, referred to as the Generative Cost Model (\textbf{GCM}), and derive a corresponding loss function. We further enhance the estimation of latent variables using variational inference and importance sampling, which reformulate our loss function accordingly. Lastly, we validate our approach through experiments on an artificial gamble simulation and two public datasets, demonstrating that our method significantly outperforms traditional techniques.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
monotonic model variational inference generative model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/e121a19fbba0a41a7db9dbe4bb782e62f9513fc5.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reformulating Strict Monotonic Probabilities with a Generative Cost Model
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission