Gradient based Causal Discovery with Diffusion Model
Causal discovery;generative models
Yes
No ethics review needed.
See weaknesses.
The authors present different experiments on synthetic and real-world datasets. The writing is clear and easy to follow.
The authors propose using diffusion models for causal discovery and searching for the DAG under continuous optimization frameworks.  The authors claim that the diffusion model has the ability to represent various functions, and the proposed causal discovery approach is able to generate graphs with satisfactory accuracy on observational data generated by either linear or nonlinear causal models.  Experiments on synthetic and real-world datasets were conducted to test the proposed method.
a. The method proposed in the paper is incremental, and it heavily relies on NoTears regularization.  The method is just a combination of diffusion model and NoTears constraint. The contribution of the paper is limited. 

b. The benefit of the method is limited. As shown in Table 1 and Table 2. 

c. The abstract is too general and does not provide sufficient information to summarize the content of the article.  It should provide insights into the proposed method.
Yes
No ethics review needed.
- I would suggest the authors to be clear about what type of functional causal model the method can handle, which is not clear from Eq. (10), (11), (12). Specifically, the paper should give a precise formulation in the form of a structural causal model $X=f_i(PA(X_i),Z_i)$.
- Is $f$ and $g$ in Eq. (10) and (11) a variable/element wise function? If so, this should be stated explicitly.
- Only CAM and DAG-GNN are compared for nonlinear models. Several other differentiable nonlinear methods could be included to strengthen the empirical studies, such as those for more general nonlinear causal models (https://arxiv.org/abs/1909.13189, https://arxiv.org/abs/1906.02226), and those for the specific causal model in Eq. (11) (https://arxiv.org/abs/1911.07420, https://arxiv.org/abs/2004.08697)

Minor:
- Proposition 1 follows similar spirit as Zhu et al. (2020), which should be made more explicitly in the main paper. Also, Proposition 1 does not provide any new insight into the optimization procedure because a typical way in differentiable causal discovery is to use augmented Lagrangian, which the paper does, so I would suggest removing this proposition.
- L56: Zheng et al. (2020) did not propose a nonparametric score function but a nonparametric DAG constraint.
Causal discovery with nonlinear relations is an important task, because it is often unrealistic to assume linear relations in practice.
This paper adopts diffusion models for differentiable causal discovery. Specifically, a specific function class is considered, where diffusion models are used to model the nonlinear causal relations. Empirical studies on synthetic and real data are provided.
- The specific nonlinear functional class considered/assumed is highly restrictive. The method cannot handle more general functional causal model, such as nonlinear additive noise models.
- The baselines considered are not adequate.
Yes
No ethics review needed.
See above
This paper is clearly written and easy to follow. The proposed approach is indeed sound and correct. Personally, although I was expecting the proposed method is to use diffusion for graph modelling (i.e. diffusion model to directly model the graph distribution), it is still interesting to see the performance if one uses diffusion model only for function modelling.
This paper proposed an approach to combine the causal discovery with the diffusion model. In particular, the author proposed to adopt a DAG-GNN approach for structural causal model but use diffusion approach to model two of the non-linear function $f$, $g$. For DAGness of the graph, they propose to use NOTEARS approach for soft but differentiable constraints. Empirically, the author evaluates on 4 synthetic datasets and two semi-synthetic ones to evaluate the performance, showing that DAG-Diffuser can outperform the baselines on some datasets.
However, I have several concerns regarding this paper. First, methodology-wise, I think the contribution is not significant. The core idea of this paper is to use diffusion model to model function $f$, $g$. Due to the generality of diffusion model (i.e. not much technical requirements if one wants to use diffusion model), it does not pose significant technical challenges of simply plug-and-play. Therefore, in terms of methodology, it is of little difference compared to using normalizing flow, invertible neural network for $f$ and $g$, which has been done before. 

Another question I have is the motivation of using diffusion model for $f$ and $g$. Is it because diffusion is flexible, and the author think that this may be helpful? If that is the case, another set of causal inference experiments can provide stronger evidence of choosing diffusion model. Since the proposed method in fact learns a structural causal model. it is straightforward to extend the framework for average treatment effect and individual treatment effect on synthetic data. Under this case, DAG-Diffuser may demonstrate stronger performance compared to baselines. 

In addition, for empirical evaluation on linear Gaussian synthetic dataset, do you use the same variance for the Gaussian noise variable? If not, the model is not identifiable, and invalidate your statements. 

Some minor comments:
1. For eq.2, is the $Y_0$ and $Y_T$ are in wrong order?

2. You should add some reference on line192. For example, [1].

3. In line 196, you mentioned that "without loss of generality, we assume....", but the invertibility of $f$ and $g$ will hurt the capacity of $f,g$. 

4. For line 201, if f is non-linear, do you still need $Z$ to be non-Gaussian? 

5. Line 261, what is $S_G$? Shouldn't it be $R_G$?

6. For related work, you should also cite some related work on discovery with soft constraints and diffusion model, like [2,3,4,5].

7. Consider introduce the method name DAG-Diffuser before the experiment section. 

8. what is $(10\alpha)/\alpha$ in line 403?


Reference

[1] Hoyer, Patrik, et al. "Nonlinear causal discovery with additive noise models." Advances in neural information processing systems 21 (2008).

[2] Rolland, Paul, et al. "Score matching enables causal discovery of nonlinear additive noise models." International Conference on Machine Learning. PMLR, 2022.

[3] Lachapelle, Sébastien, et al. "Gradient-based neural dag learning." arXiv preprint arXiv:1906.02226 (2019).

[4] Geffner, Tomas, et al. "Deep end-to-end causal inference." arXiv preprint arXiv:2202.02195 (2022).

[5] Sanchez, Pedro, et al. "Diffusion models for causal discovery via topological ordering." arXiv preprint arXiv:2210.06201 (2022).
Yes
NA
No ethics review needed.
* I noticed that CAM predicts many edges that do not exist in the true causal graphs. This could be due to the CausalDiscovery package having the tuning step turned off by default, which is crucial for the CAM algorithm. Could this be a contributing factor to CAM’s degraded performance?

* Could you clarify why DAG-diffusion’s performance is lower in linear models? Is it possibly because the diffusion model is more effective at capturing nonlinear relationships?
The paper highlights the significant advantages of using a diffusion model for causal discovery. In nonlinear models, it consistently outperforms many other baseline methods.
This paper introduces a diffusion model (denoised diffusion probabilistic model) for causal discovery and demonstrates its effectiveness in this task.
* Small typo: Line 309-310, "deep leyers"-> "deep layers". Line 183-184, "node i to node j"->"node $i$ to node $j$"


* The current form of Proposition 1 lacks rigor, raising concerns about its accuracy and possible need for revision. Here, $A$ is the weighted adjacency matrix. Notably, a sequence $A_n$ can always be constructed such that $\inf_{n} h(A_n) = 0$ (using $\inf$ here is more appropriate than $\min$), as $A$ is a weighted adjacency matrix and $h$ is continuous in $A$, implying that $h_* = 0$. Additionally, the phrase “all direct cyclic graphs in the solution space” is unclear and seems incorrect. The solution space should consist of only directed acyclic graphs, so there should be no cyclic graphs in this space. Please revise this accordingly.

* The experimental results omit the algorithm’s runtime, and results are only reported for graphs with up to 50 nodes and for ER2. Given that this paper employs a diffusion model for causal discovery, it would be helpful to include [1] as a baseline. Additional experiments would strengthen the case for the effectiveness of the diffusion model in causal discovery.

[1]Pedro Sanchez, Xiao Liu, Alison Q O’Neil, and Sotirios A Tsaftaris. Diffusion models for causal discovery via topological ordering. arXiv preprint arXiv:2210.06201, 2022.
The paper proposes to use diffusion models for causal discovery,  and searches for the DAG under continuous optimization frameworks.
@inproceedings{
anonymous2024gradient,
title={Gradient based Causal Discovery with Diffusion Model},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=l3Q0scRuT9},
note={under review}
}
Causal discovery from observational data is an important problem in many applied sciences. Incorporating a recently proposed smooth characterization of acyclicity, gradient-based causal discovery approaches search for a Directed Acyclic Graph (DAG) by optimizing various neural models. Although they show some inspiring results given certain assumptions satisfied, their capability of modeling complex nonlinear causal generative functions is still unsatisfactory. Motivated by recent advances in deep generative models, we propose to use diffusion models for causal discovery,  and search for the DAG under continuous optimization frameworks. With flexible parameter configurations, diffusion model has the ability to represent various functions, and the proposed causal discovery approach are able to generate graphs with satisfactory accuracy on  observational data generated by either linear or nonlinear causal models. This is evidenced by empirical results on both synthetic and real data.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Causal discovery generative models
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f15f1ecd43da1ee990cccf2036542dd6e1d628ab.pdf
causal reasoning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Gradient based Causal Discovery with Diffusion Model
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission