Counterfactual Learning under Rank Preservation
Counterfactual Inference;Causal Inference;Identifiability
Yes
No ethics review needed.
- What is the intuition behind the definition of the introduced loss function? Could the authors please elaborate?
- How could the loss function and the method handle discrete outcomes?
- The paper states the asymptotic unbiasedness of the estimator. However, there are no statements on finite-sample performance. How does the kernel-smoothing affect the finite-sample performance (for different variable types of z)? 
How does it compare to a discretized version (of the potentially continuous Z)? In the latter case, one would not need to perform smoothing.
- The method needs to learn the propensity function before the counterfactual estimation. How does the propensity estimation error propagate to the final counterfactual estimation error?
- Line 369: What is meant by "robustness of our method"?
- There are some typos in the text, and some citations are mixed up (unintentionally).
- The paper proposes a fundamental approach to counterfactual inference, an important topic, and can be a basis for future research in the field of counterfactual inference.
- Although the proposed approach is simple, it allows for a slight release of the stringent assumption in counterfactual inference.
- The paper is very well written and motivated.
The paper proposes a new method for counterfactual estimation. The method needs slightly less stringent assumptions than existing approaches.  Based on a novel convex loss function, the paper proposes a kernel-based counterfactual estimator and shows its unbiasedness.
- The proposed rank-preservation assumption is still very strong and unrealistic in practice.
- The mathematical notation could be introduced more thoroughly (line 87).
- The proposed method (and the loss function) only apply to continuous outcomes. However, this is not stated in the paper. Furthermore, the paper even argues with discrete outcomes (line 227).
- The related work section mostly discusses and cites works irrelevant to the topic (e.g., the paragraphs on CATE estimation and on the applications). 
- The empirical evaluation is not suitable for evaluating the proposed method. It evaluates the method for estimating treatment effect (and compares it to baselines for treatment effect estimation). 
However, the proposed method tackles counterfactual estimation. Although these two topics are obviously very connected, a proper evaluation of the main task of the method and a comparison with proper baselines for this task is necessary.
Yes
No ethics review needed.
Please see Weaknesses.
(1) The paper is clearly written and easy to read. It provides a comprehensive review of previous work on counterfactual learning, identifies gaps, and discusses similarities and differences in assumptions and estimation strategies compared to prior research.

(2)  The proposed loss function does not require prior estimation of the structural causal model, nor does it assume that the two conditional quantile models are identical. Additionally, it does not require explicit estimation of a different quantile value for each individual.
The paper introduces a rank preservation assumption to identify counterfactual outcomes. Building on this foundation, it proposes a loss function that yields an unbiased estimator, and further develops a kernel-based estimator for empirical estimation.
(1) The main concern is the validity of the rank preservation assumption. Both the identification and estimation methods rely on this key assumption, yet the paper does not provide any testing method or sensitivity analysis for it.

(2) Continuing from (1), we would like to see (i) additional synthetic experiments where rank preservation is violated to assess the performance of the proposed estimation strategy, and (ii) a method for evaluating the rank preservation assumption in real data applications.

(3) The paper only demonstrates consistency for the proposed loss function. Given that the ultimate target estimand is the counterfactual outcome, we would like to see statistical inference results, including consistency and asymptotic variance, for the counterfactual outcome.

(4) The proposed loss function estimation involves kernel smoothing, which may introduce high variance in estimation when covariates are high-dimensional. In the current synthetic experiments, all covariates are independent of each other. We would like to see additional synthetic experiments with high-dimensional, non-independent covariates.
Yes
No ethics review needed.
See above.
The paper is well written and organized. Authors clearly list their assumptions and describe their contributions.

The related work is well-covered and it motivates the approach presented in the paper.

The technical contribution is novel. Authors identify an alternative assumption sufficient for identifying the CATE and show how it can be estimated in an unbiased way through a convex loss function.

Experimental evaluation includes many baselines that are prevalent in the literature, and the proposed approach seems to present a notable performance boost.
Authors study counterfactual inference problem and propose a novel identification/estimation strategy, demonstrating its advantages through existing methods.
The experimental results can benefit from additional discussions, such as why your method has in general higher uncertainty around its performance compared to Quantile Reg (see Table 1 and Figure 1).

Also, it is a bit strange that the results for your method has notably higher standard error for $m=5$. Why is that? Because this would have important implications on methods' utility in worst case, it is important to understand and discuss the reasons behind it. 

For instance, are there special data-generating cases where your method performs worse in particular? That does not necessarily put your method down, but it is important to identify and discuss those cases.


Misc--

Line 377 - Typo: "Prepossessing"

Table 1 - CFQP has better out-sample PEHE perf.
Yes
No ethics review needed.
Apart from the comment/question in the weakness section, I have the 
following question regarding the learning algorithm guarantee: 
1. It appears that Theorem 5.3 applies to a fixed value of t, which does not imply the 
guarantee on minimizer of $\hat R_{x'}(t \mid x,z,y)$. If that is the case, I wonder if 
the results can be generalized to data-driven t?
2. Does one need to solve an optimization problem for each $(x,z,y)$?
This paper considers a very challenging and interesting question and offers a thought-provoking solution (the new identification scheme). There is an exhaustive literature review and comparison with existing work.
This paper proposes a new identification condition, under which the individual counterfactual can be identified. The condition requires the rank of counterfactuals to be in accordance with the observed outcomes. The authors then provide an algorithm for learning the counterfactuals via empirical risk minimization. The proposed algorithm is evaluated on synthetic and real data.
If I understand correctly, the new identification condition requires the rank of the observables to be *almost surely* the same as that of the counterfactuals.This appears to be a very strong assumption to me: if patient A has a better baseline than a similar patient B, then the potential outcome of patient A under treatment *has*to be better than that of patient B; but in practice, one could imagine that there can be (at least) some randomness such that this condition is violated. It would be helpful to provide more discussion on why this is a reasonable condition.
We introduce the rank preservation assumption to identify counterfactual outcomes, proposing a novel ideal loss for theoretically unbiased learning and a kernel-based estimator for empirical estimation.
@inproceedings{
anonymous2024counterfactual,
title={Counterfactual Learning under Rank Preservation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=8YsP0pBgKA},
note={under review}
}
Counterfactual inference aims to estimate the counterfactual outcome given knowledge of an observed treatment and the factual outcome, with broad applications in fields such as epidemiology, econometrics, and management science. In this paper, we propose a principled approach for identifying and estimating the counterfactual outcome.  Specifically, we introduce a simple and intuitive rank preservation assumption to identify the counterfactual outcome without relying on a known structural causal model. Building on this, we propose a novel ideal loss for theoretically unbiased learning of the counterfactual outcome and further develop a kernel-based estimator for its empirical estimation. Our theoretical analysis shows that the proposed ideal loss is convex, and the proposed estimator is unbiased. Extensive semi-synthetic and real-world experiments are conducted to demonstrate the effectiveness of the proposed method.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Counterfactual Inference Causal Inference Identifiability
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f61ea8274a2ee8254a676050da64de35ae7495fc.pdf
causal reasoning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/fe4dce187fff5029cf9cfacfe9bf8028993865d8.zip
Counterfactual Learning under Rank Preservation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission