Potential Outcomes Estimation Under Hidden Confounders
Confounders;Causal Inference;Treatment Effects
Dear reviewer aUnR,

I am the student author, and I am writing to sincerely apologize for the oversight in copying author names from the NIH website, specifically from: https://pubmed.ncbi.nlm.nih.gov/38641741/

While translating the reference from the NIH website into bibtex format, I inadvertently made errors in transcribing the full names. The original reference was provided in the following style:

Feuerriegel S, Frauen D, Melnychuk V, Schweisthal J, Hess K, Curth A, Bauer S, Kilbertus N, Kohane IS, van der Schaar M. Causal machine learning for predicting treatment outcomes. Nat Med. 2024 Apr;30(4):958-968. doi: 10.1038/s41591-024-02902-1. Epub 2024 Apr 19. PMID: 38641741. 

In the process of converting this reference to a bibtex format, I mistakenly altered some of the names. I cannot pinpoint precisely how this happened, but I want to clarify that the reference was not generated from scratch by a language model (LLM). I used tools like Writefull and Grammarly at the sentence level (as disclosed in the submission), and these may have unintentionally modified the names.
Note that all other references are correct as I just double checked them and there were no errors in the other names. 

I want to assure you that no part of the paper was generated by an LLM, aside from sentence-level edits as disclosed. 

Best,
Student Author
My bad--his title is Senior Program Chair. Somehow there are too many different titles, and I thought it was the same.
My bad--his title is Senior Program Chair
I had a discussion on email with Carl Vondrick which the ICLR webpage calls him a Senior Area Chair
I did have a discussion with Carl Vondrick<vondrick@cs.columbia.edu>
Hi Senior Author, 

I am your Senior Area Chair, and you did not have a discussion with me. Please refrain from claiming that you have had interactions that you have not. 

Best,
Your Senior Area Chair
Untrue Claim By Senior Author
I just had an email discussion with the senior program chair.

Again, you made some very serious allegations, and it was so serious that made me extremely upset.  I do not get upset by a paper being rejected (as I said we all know about the quality of the reviews in ML conferences), but these kinds of allegations are too much.

I have foreign students who are not familiar with other national names, and one of them had a couple of typos in entering a first name or two in one reference (while working very late). I am sorry if you may be one of the authors of that paper. 

Just so that everyone knows, I always re-write the entire text of the paper as students' English is not perfect.  Granted that I missed a couple of typos in one of the references but the allegation that we produced this paper with LLMs is ridiculous.

Now you say:

"Directly at the beginning of the reviewing period, I wrote a comment to the program committee about potential ethical concerns. I thought that this comment would become public during rebuttal time. Therefore, I referred to the "comment above" in my review. As this is not the case, I will repost my comment below."

What makes you think a couple of typos make a paper LLM generated while your own review is so incoherent?

In anyways, my comment that your review was LLM generated was out of being provoked by your allegations. I am sorry about that. However, I maintain that your review is incoherent\ and has many issues. That said, I appreciate the time that you put into it. I do not claim to be a better reviewer than you are

Best wishes

Senior Author
I just had a discussion with Senior Program Chair
You made very serious allegations. Now you say there were some typos.
Since you do not apologize and do not seem to retract your statements/reviews, I will have to write to the TPC chairs and demand an investigation.

Your review is non-sense, but as I said, I am used to nonsense. Your comments are preposterous.

Senior author
Is this an apology?
Dear authors,

dear PC, SAC, Ac,

If there is reason to believe my review is machine-generated or in some way uncoherent, I am happy to explain my comments in more detail. Directly at the beginning of the reviewing period, I wrote a comment to the program committee about potential ethical concerns. I thought that this comment would become public during rebuttal time. Therefore, I referred to the "comment above" in my review. As this is not the case, I will repost my comment below. 

Of note, the reviewers make **the same mistake** in their official answer to my review, accusing me of providing unsubstantiated, misleading, and unethical reviews. The next time the authors make such a strong statement I would highly encourage them to check on the correctness of their answer.

*Dear AC, SAC, PC,*

*While reviewing the paper, some ethical concerns regarding truthful scientific practice and the misuse of large language models arose. I would like to ask your opinion on this matter and the potential consequences.*

*Specifically, the concerns arose when reading this version of a reference stated in the submitted paper:*

*Stefan Feuerriegel, **Daniel** Frauen, **Viktoria** Melnychuk, **Julian** Schweisthal, **Katharina** Hess, Alicia Curth, **Sebastian** Bauer, Niki Kilbertus, Isaac S Kohane, and Mihaela van der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30(4):958–968, 2024.*

*The correct reference should read:*

*Stefan Feuerriegel, **Dennis** Frauen, **Valentyn** Melnychuk, **Jonas** Schweisthal, **Konstantin** Hess, Alicia Curth, **Stefan** Bauer, Niki Kilbertus, Isaac S Kohane, and Mihaela van der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30(4):958–968, 2024.*

*I can only explain this uncommon mistake through the use of LLMs. I am aware that the stated issue is not extremely severe. However, it raises questions regarding the misuse of LLMs in other parts of the work.*

*Best regards,*

*Reviewer aUnR*
Answer to unjustified accusation
Dear TPC Chairs, Senior Area Chairs, Area Chairs

I am the senior author of this submission. I am writing to report a potential violation of ethics codes by reviewer aUnR.
As you can see the reviewer makes the allegation that the citations of our paper are non-existent, fake and machine generated and that the paper in part may be LLM generated. 

In the response that I just posted, we gave every single citation of the paper and provided links to every citation. This demonstrates the falseness of allegations made by the reviewer.

While we value constructive criticism, the reviewer makes several baseless claims, vague and non-sensial critiques, and the review has many frequent inaccuracies. We believe that these issues hinder the academic integrity of the review process. 

Note that I am a seasoned researcher and I and my students are used to low quality reviews and materially wrong reviews at ML conferences, but I find the reviewer aUnR baseless allegations very outrageous.

I am writing to you to demand an ethics investigation into reviewer aUnR  actions in this matter. From reading his/her/their review, I suspect that this review may be machine generated in part (as it is very non-coherent).

I will write to you directly (outside of the Openreview platform) to formally demand an investigation unless the reviewer aUnR retract his/her/their review.

Best regards, 
Senior Author
Demanding Investigation of Reviewer aUnR for Ethical Violation
As shown, there is no evidence to support your claim of false references. Furthermore, suggesting that more of our manuscript may be generated by an LLM, based solely on this incorrect assertion, is speculative and lacks any concrete basis.

Incorrect Line References:

Several of the reviewer’s comments are linked to irrelevant line numbers, making them difficult to address:
Line 58: You reference this line to be a paragraph headline, yet it points to Figure 1.
Line 70: Your comment about missing references is directed at a figure caption, where no references are applicable.
Line 35: There is no discussion of consistency at this line, contrary to your claim.
These errors suggest a significant lack of attention to the manuscript or confusion about its content.

Vague and Unhelpful Critiques:
You state that our presentation "contains errors, hindering easy understanding of the line of thought," without specifying any instance of such errors. This type of feedback is too vague to be actionable, and we ask for specific examples to improve our work meaningfully.

Technical Misunderstandings:
You question the definition of $g$ in Proposition 3.2, although it is explicitly defined within the proposition. This indicates either a lack of familiarity with standard mathematical notations or insufficient attention to the manuscript. 



Conclusion:

While we value constructive criticism, your review includes several baseless claims, vague critiques, and frequent inaccuracies. We believe that these issues hinder the academic integrity of the review process. We are used to low quality reviews and materially wrong reviews at ML conferences, but your baseless allegations are outrageous.  

We request that either you immediately show some remorse, and enter a serious apology or else we will write to technical committee with the screenshot of your dishonest comments included and request an ethics investigation in this matter.

 Best regards,
The Authors.
Continuation Response to your wrong, unethical, careless and probably machine generated allegations/review (2)
[15] Tobias Hatt, Daniel Tschernutter, and Stefan Feuerriegel. Generalizing off-policy learning under sample selection bias. In Uncertainty in Artificial Intelligence, pp. 769–779. PMLR, 2022b.
Link: https://proceedings.mlr.press/v180/hatt22a.html

[16] Miguel A Hernán and James M Robins. Causal Inference: What If. Chapman & Hall/CRC, BocaRaton, 2020.
Link: https://www.hsph.harvard.edu/miguel-hernan/wp-content/uploads/sites/1268/2024/01/hernanrobins_WhatIf_2jan24.pdf

[17] Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217–240, 2011.
Link: https://www.tandfonline.com/doi/abs/10.1198/jcgs.2010.08162

[18] Guido W Imbens. Causal inference in the social sciences. Annual Review of Statistics and Its Application, 11, 2024.
Link: https://www.annualreviews.org/content/journals/10.1146/annurev-statistics-033121-114601

[19] Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
Link:https://www.cambridge.org/core/books/causal-inference-for-statistics-social-and-biomedical-sciences/71126BE90C58F1A431FE9B2DD07938AB

[20] Nathan Kallus and Angela Zhou. Confounding-robust policy improvement. Advances in neural information processing systems, 31, 2018.
Link: https://papers.nips.cc/paper_files/paper/2018/hash/3a09a524440d44d7f19870070a5ad42f-Abstract.html

[21] Nathan Kallus, Aahlad Manas Puli, and Uri Shalit. Removing hidden confounding by experimental grounding. Advances in neural information processing systems, 31, 2018
Link: https://papers.nips.cc/paper_files/paper/2018/hash/566f0ea4f6c2e947f36795c8f58ba901-Abstract.html

[22] Nathan Kallus, Xiaojie Mao, and Angela Zhou. Interval estimation of individual-level causal effects under unobserved confounding. In The 22nd international conference on artificial intelligence and statistics, pp. 2281–2290. PMLR, 2019.
Link: https://proceedings.mlr.press/v89/kallus19a.html

[23] Robert J LaLonde. Evaluating the econometric evaluations of training programs with experimental data. The American economic review, pp. 604–620, 1986.
Link: https://www.jstor.org/stable/1806062

[24] Haoxuan Li, Kunhan Wu, Chunyuan Zheng, Yanghao Xiao, Hao Wang, Zhi Geng, Fuli Feng, Xiangnan He, and Peng Wu. Removing hidden confounding in recommendation: a unified multi-task learning approach. Advances in Neural Information Processing Systems, 36, 2024.
Link: https://openreview.net/forum?id=4IWJZjbRFj

[25] Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. Bounds on representation-induced confounding bias for treatment effect estimation. In The Twelfth International Conference on Learning Representations, 2024
Link: https://openreview.net/forum?id=d3xKPQVjSc.

[26] Karl Popper. The logic of scientific discovery. Routledge, 2005.
Link: https://www.taylorfrancis.com/books/mono/10.4324/9780203994627/logic-scientific-discovery-karl-popper-karl-popper

[27] Jonathan Richens and Tom Everitt. Robust agents learn causal world models. In The Twelfth International Conference on Learning Representations, 2024.
Link: https://openreview.net/forum?id=pOoKI3ouv1

[28] Paul R. Rosenbaum. Observational Studies. Springer, New York, 2nd edition, 2002.
Link: https://link.springer.com/book/10.1007/978-1-4757-3692-2

[29] Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41–55, 1983.
Link: https://academic.oup.com/biomet/article/70/1/41/240879

[30] Jonas Schweisthal, Dennis Frauen, Mihaela Van Der Schaar, and Stefan Feuerriegel. Meta-learners for partially-identified treatment effects across multiple environments. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 43967–43985. PMLR, 21–27 Jul 2024.
Link: https://proceedings.mlr.press/v235/schweisthal24a.html

[31] Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning, pp. 3076–3085. PMLR, 2017.
Link: https://proceedings.mlr.press/v70/shalit17a.html

[32] Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. Advances in neural information processing systems, 32, 2019.
Link: https://papers.nips.cc/paper_files/paper/2019/hash/8fb5f8be2aa9d6c64a04e3ab9f63feee-Abstract.html

[33] Jeffrey A Smith and Petra E Todd. Does matching overcome lalonde’s critique of nonexperimental estimators? Journal of econometrics, 125(1-2):305–353, 2005.
Link: https://www.sciencedirect.com/science/article/pii/S030440760400082X

[34] Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228–1242, 2018.

Link: https://www.tandfonline.com/doi/full/10.1080/01621459.2017.1319839
Continuation Response to your wrong, unethical, careless and probably machine generated allegations/review (1)