G-Transformer for Conditional Average Potential Outcome Estimation over Time
causal inference;potential outcomes;treatment effects;G-computation;time-varying confounding;medicine
Yes
No ethics review needed.
See weaknesses above.
- The paper provides a clear, detailed description of model architecture.

- The authors conduct extensive experiments with synthetic and semi-synthetic datasets, demonstrating GT's effectiveness under different levels of confounding settings.

- The code is provided.
The paper studies conditional average potential outcome estimation over time. The authors propose a model called G-Transformer for addressing two primary challenges in the field: adjustment for time-varying confounders and low variance in estimates. The authors validate their approach through experiments on synthetic and semi-synthetic data, demonstrating improvements over current baselines.
- The contribution of this paper is somewhat limited. 
  * From the methodology perspective, the proposed method's improvement over existing approaches appears marginal. The primary contribution is exploring the transformer backbone in treatment effect estimation, which has been widely investigated in existing work (static setting [1] and dynamic setting [2]).
  * From the application perspective, the paper claims that their model can help personalize decision-making from patient trajectories in medicine. However, it lacks discussion on how the model's estimates would concretely translate to real-world decision support. Furthermore, the assumptions on which the model relies (e.g., positivity, ignorability) are restrictive and may not hold robustly in practical, real-world scenarios.

- While the paper claims to reduce estimation variance, it lacks a theoretical analysis of variance bounds. Given GT’s objective to minimize estimation variance, a more rigorous theoretical justification or a derivation of variance bounds would significantly strengthen this claim.

-  The paper does not consider the existence of unobserved (and potentially time-varying) confounders, which can introduce biases and affect CAPO estimation in real-world EHR settings.

- The study uses a semi-synthetic dataset derived from MIMIC-III. It would be valuable to see the model's performance on fully real-world datasets to assess its utility in practical clinical decision-making problems.

[1] Zhang, Yi-Fan, et al. "Exploring transformer backbones for heterogeneous treatment effect estimation." arXiv preprint arXiv:2202.01336 (2022).

[2] Melnychuk, Valentyn, Dennis Frauen, and Stefan Feuerriegel. "Causal transformer for estimating counterfactual outcomes." International Conference on Machine Learning. PMLR, 2022.
Yes
No ethics review needed.
Table 2 and 3 have low readability due to small font sizes.

The reviewer thinks Supplementary E.2 regarding the quality of pseudo-outcome is worth being mentioned in the main text
The study is motivated and the problem is well-defined by the research gap of the previous studies. Specifically, the authors address the limitations of the previous studies regarding G-computation to estimate CAPOs with neural networks by providing a viable solution of iterative pseudo-outcome regression, although the neural network architecture itself is similar to the existing study (Melnychuk et al., 2022).

The author provides sufficient and proper supplementary materials to explain the details of the study.
This study proposes G-Transformer to estimate conditional average potential outcomes (CAPOs) over time while addressing two major limitations of the previous studies: (1) adjustments for time-varying confounders; (2) and large estimation variance. Particularly, the authors propose a viable solution to G-computation, iterative pseudo-outcome regression, using Transformer.
The reviewer did not find any significant weaknesses.

The reviewer thinks a more principled and diverse way to evaluate the quality of the generated pseudo-outcomes will be desirable. While the experiment conducted in E.2 could empirically show the learning step is sensitive to the noise of the generated pseudo-outcomes, it might not be sufficient to show the quality of the generated pseudo-outcomes as it is titled. An ablation study that separates each step (i.e., another step is frozen when one step is trained) might be useful for a controlled experiment to measure the sensitivity of the performance in terms of how much the model is trained to generate the pseudo-outcomes.
Yes
No ethics review needed.
My suggestions correspond to the two weaknesses above. I think comparison to prior work that already proposed the combination of transformer and temporal-difference learning/pseudo-outcome regression is absolutely crucial; literature review related to pseudo-outcome regression ([1][3][4] and other related work) should be added, and contribution of this work should be properly and correctly stated.
1. Paper is well written, problem formulation, architecture descriptions and the presentation of learning procedures are all clearly stated and very easy to follow.

2. The results look compeling. Table 2 and 3 showed noticable improvement of the proposed model over the compared benchmarks.
This work proposed a framework to estimate conditional average potential outcome over time using a combination of transformer and iterative pseudo-outcome regression. The framework is tested in experiments on a synthetic dataset based on tumor growth model and a semi-synthetic dataset based on MIMIC-III, and is shown to outperform several benchmark models on these two datasets.
1. The novelty of this work is the most important concern to me. The combination of transformer and temporal-difference learning (i.e. pseudo-outcome regression as called by the authors) are already proposed in [1] for the estimation of CAPO (on top of that, [1] also went one step further and proposed the adjustment for marginal causal parameters). Besides, the heterogeneous token transformer architecture in [1] appears to be a more clean and intuitive solution as opposed to the proposed architecture in my judgement. The proposed transformer architecture is rather trivial and does not significantly differ from [2]. At the very least, the author should compare their architeture to [1], show experimental results and properly state their contributions compared to prior work.

2. The paper has a very weak description of prior work related to pseudo-outcome regression in my opinion. Not only is the exact same procedure temporal-difference learning proposed before as stated above, it is widely used in causal inference and reinforcement learning known as sequential regression [3] and Q-learning [4]. The omission of these works by the authors made it look like pseudo-outcome regression is a novel contribution of their own.

[1] Shirakawa, Toru, et al. "Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer." International Conference on Machine Learning. PMLR, 2024.

[2] Melnychuk, Valentyn, Dennis Frauen, and Stefan Feuerriegel. "Causal transformer for estimating counterfactual outcomes." International Conference on Machine Learning. PMLR, 2022.

[3] van der Laan, M. J. and Gruber, S. Targeted Minimum Loss Based Estimation of Causal Effects of Multiple Time Point Interventions. The International Journal of Biostatistics, 8(1), 2012.

[4] Chebotar, Yevgen, et al. "Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions." Conference on Robot Learning. PMLR, 2023.
Yes
No ethics review needed.
- It seems that the GT approach has a clear advantage over existing approaches when $\tau \ge 2$. Could you provide scenarios where estimating CAPO over time ($\tau \ge 2$) has clear clinical benefits?
- Could you quantify the variance of the baseline approaches, for example, using the coefficient of variation?
- Could you provide additional results on the precision in the estimation of heterogeneous effects, average treatment effects, and qualitative analysis measures of over- or underestimation of outcomes?
- The paper addresses the important and impactful real-world problem of estimating CAPO over time under time-varying confounding and treatment from observational data.
- The proposed GT, iteratively generates pseudo-outcomes by reformulating the G-computation formula as recursive conditional expectations, which seems interesting.
- Experimental results on synthetic and semi-synthetic datasets demonstrate that the proposed approach outperforms the baselines in terms of RMSE, especially under conditions of high time-varying confounding and longer CAPO estimation horizons.
- The code and data are publicly available, making it easy to reproduce the results.
The paper proposes the G-transformer (GT) approach, which combines the G-computation formula and a multi-input transformer to estimate conditional average potential outcomes (CAPO) over time, given time-varying confounding and treatment observational data. GT iteratively generates pseudo-outcomes by reformulating the G-computation formula as recursive conditional expectations. Experimental results on synthetic and semi-synthetic datasets demonstrate that the proposed approach outperforms the baselines in terms of average root mean squared error (RMSE).
- The novelty is quite limited; the proposed approach could be considered a simple extension of the causal multi-input transformer (Melnychuk et al., 2022) with pseudo-outcome predictions.

- While the paper claims to be the first to use pseudo-outcomes for CAPO estimation over time, several works have already explored the use of pseudo-observations in causal inference and survival analysis, including [1,2,3]. I encourage the author(s) to discuss these works and highlight key differences with their proposed approach.

- The paper asserts that related works, such as G-Net, which properly adjust for time-varying confounding, result in high estimation variance without providing substantial experimental or theoretical results to support this claim. Only RMSE values with error bars are provided (Tables 2 and 3), which seems insufficient given that this is a central claim of the GT approach.

- The experimental results seem underwhelming. Only one semi-synthetic dataset is considered, and RMSE is provided to quantify performance. I encourage the author(s) to explore evaluations tied to meaningful clinical measures, including precision in the estimation of heterogeneous effects, average treatment effects, and qualitative analysis measures of over- or underestimation of outcomes.

- Clarity: The notation and writing need improvement for clarity and readability. The use of both $U$ and $H$ to denote the random variables $\{Y_t, X_t, A_t\}$ does not seem necessary. The paper seems to refer to $H$ throughout, and it's unclear why $U$ was introduced. The definitions for $\bar{Y}$, $\bar{X}$, and $\bar{A}$ are not provided. The recursion could be made more explicit by combining Equations (8) and (9) and providing a simple example, e.g., $\tau = 3$. The paper's use of $\delta$, $\tau$, and $t$ for indexing time needs further simplification. Additionally, the definition for $\tau$ should be provided in the *Setup* section.


**References**
- [1] Andersen et al. (2017), "Causal inference in survival analysis using pseudo-observations", Statistics in medicine.
- [2] Andersen and Perme (2010), "Pseudo-observations in survival analysis", Statistical methods in medical research.
- [3] Chien-Lin et al. (2022), "Causal inference for recurrent event data using pseudo-observations", Biostatistics.

**Minor**
- Table 1: I encourage the author(s) to include citations to related works for improved readability.
- Notation overload:  $\phi$  is used to parameterize both functions $g(\cdot  )$ and $z( \cdot)$.
- Typos: Line 140 should be  $U_{t+\tau}$ remove the 'minus'.
- I encourage the author(s) to increase the font size for tables and figures to be consistent with the main text.
- The definition for $\delta$ in Line 244 is not consistent with that in Line 267.
We develop a novel neural architecture for low-variance estimation of conditional average potential outcomes over time with proper adjustments through regression-based iterative G-computation.
@inproceedings{
anonymous2024gtransformer,
title={G-Transformer for Conditional Average Potential Outcome Estimation over Time},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=XUJcsLvpaQ},
note={under review}
}
Estimating potential outcomes for treatments over time based on observational data is important for personalized decision-making in medicine. Yet, existing neural methods for this task either (1) do not perform proper adjustments for time-varying confounders, or (2) suffer from large estimation variance. In order to address both limitations, we introduce the G-transformer (GT). Our GT is a novel, neural end-to-end model which adjusts for time-varying confounders, and provides low-variance estimation of conditional average potential outcomes (CAPOs) over time. Specifically, our GT is the first neural model to perform regression-based iterative G-computation for CAPOs in the time-varying setting. We evaluate the effectiveness of our GT across various experiments. In sum, this work represents a significant step towards personalized decision-making from electronic health records.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
causal inference potential outcomes treatment effects G-computation time-varying confounding medicine
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/d3d2268ba79b22ce04308fa8f14efdf129e87edd.pdf
causal reasoning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
G-Transformer for Conditional Average Potential Outcome Estimation over Time
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission