Feynman-Kac Operator Expectation Estimator
Expectation Estimator;Diffusion bridge model;MCMC;Physically Informed Neural Networks;Minimum Wasserstein Estimator
Thank you for reading our paper in detail! Below are my responses to some of your statements. If any of them are not precise enough, feel free to ask more questions.

The main contribution of this paper is a different interpretation of the FE formula, which can enhance the performance of most diffusion models (samplers) in estimating expectations. Blechschmidt and Ernst (2021) is a classic method for solving the FK equation, meaning that we can solve high-dimensional PDEs by simulating SDEs (which typically do not have dimensionality issues) and then computing expectations. This is just one way of utilizing the PDE. We shift this approach by assuming that we know the SDE is a diffusion (diffusion bridge) model, and accordingly, we can naturally express the corresponding PDE. By using a different method to solve the PDE, namely PINN, we can obtain the expectation of the SDE at the terminal point, which corresponds to the solution at the initial time of the PDE.

This method avoids the need for LLN and ETMC to compute the mathematical expectation, and we often only need a small number of points to achieve better results. This is because the PDE integrates all the information of the SDE, and the SDE contains information such as the density, for example, in Langevin dynamics and score functions in score-based SDEs. 

Therefore, our validation experiments are centered around expectation estimation, rather than the sampler(MCMC) and PDE.  

Expanding the Scope of Expectation Estimators can be specifically referenced in line 415. In other words, we can integrate a large number of SDE samplers to expand the range of  $P$ , which may not even be a stationary distribution in MCMC. This weakens the conditions on $f $, as a larger Lipschitz constant for $f$ often leads to larger variance.

Integrating MCMC into a unified framework has been recently achieved by many diffusion models, including our method, as seen in [1][2][3]. This represents the latest direction in current technology. However, there is a lack of corresponding work on efficiently obtaining expectations from SDEs, as most estimators still rely on ETMC and LLN.

[1] Vargas, Francisco, Will Sussman Grathwohl, and Arnaud Doucet. "Denoising Diffusion Samplers." The Eleventh International Conference on Learning Representations.

[2]Richter, Lorenz, and Julius Berner. "Improved sampling via learned diffusions." The Twelfth International Conference on Learning Representations.

[3]Grenioux, Louis, et al. "Stochastic Localization via Iterative Posterior Sampling." Forty-first International Conference on Machine Learning.

Our experiments demonstrate that we have obtained more accurate estimates of the partition function with fewer samples. Sampling algorithms and expectation estimators are fundamentally different methods. As we have emphasized, our focus is not on the MCMC method itself, but rather on whether MCMC + FE outperforms traditional methods. Our experiments have already shown that this is the case. Replica-exchange Monte Carlo is merely a sampling method and does not provide an expectation estimator. Moreover, the estimator will still be influenced by the conditions on $ f $.

Questions

Can the author disentangle their works from past literature on neural SDE and Feynman-Kac's use of averaging observables?

Can the authors provide evidence that their universal sampler outperforms the existing techniques, including those in Blechshmidt et. al.(2021)? Can they provide some canonical well-excepted benchmark at which they excel over others?

Answers see above.We welcome and accept any suggestions for improving the expression and clarity of the paper.
Responses
What we propose in this paper is not a sampler or a generative model, but rather a heuristic algorithm that can enhance the performance of most existing SDE-based samplers in estimating expectations.

Theoretically, we have provided the convergence results for our diffusion bridge model. The diffusion bridge model is merely a sampler designed to reduce the total training effort of the PINN. We reverse-engineer the FK equation, which is itself a conceptual innovation. Most of our past work focused on solving high-dimensional PDEs, namely using MCMC / SDEs and then compute the expectation to obtain the solution of the PDE. Our approach, however, is to directly solve the PDE to obtain the MCMC expectation. When the SDE is a diffusion model-based sampler, we can obtain the expectation of the target distribution without relying on the LLN or ETMC.

Samplers using the $W_2$ distance are quite common, such as those based on optimal transport methods. For high-dimensional cases, Since the dimension is hidden in our Theorem 2.7, a classical convergence result is given in [1].

[1] Fournier, Nicolas, and Arnaud Guillin. "On the rate of convergence in Wasserstein distance of the empirical measure." Probability theory and related fields 162.3 (2015): 707-738.

We compare the SOTA expectation estimators and demonstrate the performance of PINN in the high-dimensional case with $d = 225$, as shown in the experiment in line 1029.

How does your algorithm compare with other neural ODE/SDE, diffusion models, bridges models in the literature?

Our goal is to estimate the expectation, not to sample. Therefore, our experiments focus more on the combination of the sampler + FE and sampler + ETMC/LLN, with the choice of sampler being diverse.
Responses
Q1 The scalability of the algorithm w.r.t. dimension is not verified sufficiently. d=20 is too small. There are no real-world simulations.

A1 In fact, we demonstrated the effectiveness of this method in high-dimensional cases, such as 225 dimensions, in the ising model. A detailed explanation of this experiment can be found in line 1077.

Q2 The authors criticize the large variance issue by the MCMC method but fail to justify theoretically why the proposed method yields a lower variance. The empirical support is limited. 

A2 Theoretically, concentration inequalities can show that a larger Lipschitz constant will lead to a larger variance, which can be referenced in [1]. On the application level, the precision of the sampler also affects the variance. For example, using the Metropolis Adjusted Langevin Algorithm for correction. Our two experiments validate these two cases. First, for the ISING model case, the complexity of $f$ is the main factor. In the second case, we use the unadjusted Langevin Algorithm to obtain better results, without changing the structure of the sampler.

The metric used in Girsanov's theorem is the KL divergence, but KL divergence does not have the properties of $W_2$. Currently, we are more focused on $W_2$. Additionally, Girsanov's theorem cannot be applied when the diffusion coefficients are inconsistent.


[1] Gobet, Emmanuel. Monte-Carlo methods and stochastic processes: from linear to non-linear. Chapman and Hall/CRC, 2016.

Q4 I don't know why and when MCMC is required to impose complex constraints on the distribution and performance function. Some references are suggested.

A4 In fact, the example in the ISING model illustrates this point. In a more general case, when we want to solve an energy model such as $p(x;\theta) = \frac{\exp(-U(x,\theta))}{Z(\theta)}$, the partition function $Z(\theta)$ is intractable, especially when the state space of $x$ is discrete. This type of model is commonly found in large language models. Below are examples of handling $Z(\theta)$. In addition, in statistical inference, for example, when estimating the parameters of random graph models, this issue arises. While we can use MCMC sampling distributions, we cannot obtain an exact expression for the distribution due to the presence of the partition function.


[2] Xu, Minkai, et al. "Energy-Based Diffusion Language Models for Text Generation." arXiv preprint arXiv:2410.21357 (2024).

[3] Rafailov, Rafael, et al. "Direct preference optimization: Your language model is secretly a reward model." Advances in Neural Information Processing Systems 36 (2024).


Q5 I don't see when MCMC is not the optimal decoding method. It appears to me that burn-in is not a significant limitation and only affects the performance on a negligible scale and can be easily fixed via a large stepsize in the beginning for warm-up.

A5 The reason for this is quite simple: in diffusion models, $X_T$ does not necessarily have to be a stationary distribution. For example, methods like normalized flow methods and many diffusion model-based samplers can all use FE to solve for expectations without relying on ETMC and LLN. For MCMC objectives, diffusion bridge modelling is a more efficient approach [1].


[1] Vargas, Francisco, Will Sussman Grathwohl, and Arnaud Doucet. "Denoising Diffusion Samplers." The Eleventh International Conference on Learning Representations.

We appreciate the suggestion to discuss the limitations.
Responses
Q1 The Feynman–Kac model (Algorithm 2) with the PINN solver lacks a convergence or error estimate, which would be valuable for assessing its accuracy and reliability. 
A1 Yes, this is indeed a problem. We provide some empirical results, but theoretically, we have actually cited the relevant convergence results [1] for such PDEs in line 380.

[1]Tim De Ryck and Siddhartha Mishra. Error analysis for physics-informed neural networks (pinns)
approximating kolmogorov pdes. Advances in Computational Mathematics, 48(6):79, 2022.

Q2 In the experiments, the authors claim that "using fewer points on the Markov chain achieves higher accuracy in approximating expectations." However, it is unclear if this result generalizes beyond the specific example provided, as it appears quite context-dependent.

A2  We mainly emphasize the statement in line 485. The main text only provides a rough result, and more details about this experiment can be found in the Appendix 9.1.

Questions about conditions on functions

A The function condition we are referring to is in the context of expectation estimators. In this field, Lipschitz functions will significantly affect the estimator's equation. For more details, you can refer to Section 2.4.4 in [1] and other concentration inequalities. Our assumption only requires the function to belong to $C^2$ and does not impose any restrictions on the Lipschitz constant. 

Potential limitations this might introduce for functions that are not in C^2, this could lead to potential issues in the convergence of PINN training, such as the generation of large gradients at the boundaries for functions that are not in $C^2$.


[1] Gobet, Emmanuel. Monte-Carlo methods and stochastic processes: from linear to non-linear. Chapman and Hall/CRC, 2016.
Responses
Thank you very much for your careful reading and for pointing out the unclear expressions in the paper. We have made the necessary corrections in a revised version. It has now been updated. 

Here are some responses to the questions: 
Q1 In Theorem 2.1, what does "Linear growth" mean?
A1 On line 815, we provide the definition in the appendix.

Q2 In Assumption 2.4, what does "D is the metric of the parameter" mean?

A2 $P$ is a parameter of a triplet. We define the metric, which can be considered in the product space, for example, the first for $X_0$ is the Euclidean 2-norm of vectors, the second for $b$ is the $L_2$ norm, and the third for $\sigma$ is the Fubinius norm.

Q3  In Theorem 2.6, should "we exist" be "there exists a set"?
A3  yes, your phrasing is more precise.

Q4 In Algorithm 1, can the authors clarify what is the main difference between X_i and $X_t$ The distinction is not clear to me. 
A4  This is a typo, X_i should be $X_{t_i}$. We have revised the corresponding paragraph.

Q5 How are the integrals in Algorithm 2 computed? ...

A5 Sure. First, to solve the FK equation, we need $(x_0, b, \sigma)$, which are obtained from the diffusion bridge model described earlier. Of course, these can also be obtained from any other diffusion model, as shown in the first table of the appendix. Next, we need to sample the PDE at $(x, t)$, which means we will first run the SDE. In our previous diffusion bridge model, this is based on the SDE solver, so we already have the samples $(X_t, t)$ that have been run. Then we compute the residual terms corresponding to these points, which are equations (13) and (14), and add them to the training. After training is complete, the solution $u(x,t)$ at the initial time is the mathematical expectation of the terminal distribution, which is the integral. The reason for doing this can be referenced in 3 Discussion. Overall, by doing so, we can maximize the performance of the diffusion model in estimating the mathematical expectation, without relying on the law of large numbers or ergodic theorems. For example, we do not require that $X_T$ follows a stationary distribution, and we have extended the range of $f$ in the expectation estimator.

Q6 How does the computational cost of this approach compare to other MCMC approaches?

A6 We believe there may be some misunderstanding regarding the main contribution of this paper. We are not making incremental improvements to diffusion models (MCMC), although we propose a sampler. What we aim to express is that the expectation estimator, i.e., (diffusion model/MCMC) + FE, will provide better estimation performance than (diffusion model/MCMC) + (LLN/ETMC). Therefore, all our usage revolves around expectation estimation, such as estimating the partition function, etc. 

The computational cost is divided into two parts: The first part involves solving the diffusion bridge. We use Sinkhorn's method to solve the optimal transport problem. This part is relatively fast, and you can refer to the code in our supplementary materials. Of course, if we have a pre-trained diffusion model, this computation can be completely ignored, or we can use explicit SDE samplers such as Langevin sampling. The second part, which involves training, is computationally more expensive, so we have put a lot of effort into this area. Please refer to line 275. First, we control the number of time steps for the diffusion bridge and the total number of training points. Second, we consider diagonal parametrization of $\sigma$.

The last question is very valuable, and we need to provide some clarification. First, what we aim to express is that sampling is not the same as expectation estimation, because the expectation estimator is also affected by $f$ and the quality of the sampling. However, we can avoid this issue compared to traditional expectation estimators. The influence of $f$ can be illustrated using the ISING model example, where $f$ leads to a large bias due to being an exponential function. A more intuitive example is the Langevin sampling example in the appendix, where the sampler we use is biased, but our method can reduce this bias without modifying the sampler itself. 

This also answers your question: if we have some points sampled from the MCMC stationary distribution, we have three possible computation methods. The first is directly computing the average. The second method is to generate more points using the diffusion bridge and then compute the average. The third method is using the diffusion bridge + FK equation. We have shown that the third method is better than the first two. The main reason is that in the diffusion bridge, the parameters $b$ and $\sigma$ include all the information about the distribution. For example, in Langevin sampler, the $b$ is composed of the density($\frac12 \nabla_x \log p(x)$). Solving the PDE can better integrate this information.

If you have any questions, feel free to discuss further.
Responses
Yes
No ethics review needed.
In Theorem 2.1, what does "Linear growth" mean?

In Assumption 2.4, what does "D is the metric of the parameter" mean?

In Theorem 2.6, should "we exist" be "there exists a set"?

In Algorithm 1, can the authors clarify what is the main difference between $X_t$ and $X_i$? The distinction is not clear to me. 

How are the integrals in Algorithm 2 computed? Are the authors able to evaluate the integrals explicitly? If so, they should indicate how and why they are able to do so.

How does the computational cost of this approach compare to other MCMC approaches?

One of the main criticisms posed about MCMC methods is that they are not optimal since they spend quite some time in the burn-in phase (lines 56, 82). However, for the proposed method to work the authors assume that they have access to samples from a distribution that are obtained via an MCMC, that has already gone through a burn-in period (lines 188, 192). How can the authors support their claim that this method is better (line 107) than MCMC if it is still spending a similar amount of samples in burn-in?
The goals stated in the introduction are bold, and quite interesting. Obtaining results in this line would prove quite useful in general for ML and statistics.

I appreciate that the authors include small introductions for the Euler-Maruyama method and physics informed neural networks in the appendix. However their existence should be indicated in the main text.
Doing MCMC is hard (time consuming, and somewhat wasteful because of the burn-in period). The authors propose a post-processing method using the samples from some MCMC procedures, which admit an Itô decomposition, to approximate moment estimates of the desired sampling density. They use a denoising technique based on physics informed neural networks as part of the post-processing mechanism.
Presentation is bad throughout. There are plenty of typos. The authors do not use parenthetical citations and instead insert them in the text which makes for a less pleasant reading experience. 

The notation introduced in line 232 definitely needs improvement, I do not understand which side is supposed to be the one that will be used later. Even then, it is unclear what is being defined, as there are two definitions for $\hat{\mu}\_{t\_i}$ .

In Assumption 2.2, it is not clear what $\mu^{\mathcal{P}_\theta}$ means.

The notation in Algorithms 1 and 3 should be introduced before the algorithms. For example, it is not clear where $Y_T$ comes from, and why it is required.

Table 1 is impossible for me to parse. I invite the authors to mimic the conciseness of their own Table 5 for summarizing the numerical results.

There is a link to GitHub page, which has been confirmed to not belong to the authors, but I do not see a good reason for why the authors would want to include a link to a GitHub that is not theirs. Usually a citation to the original article is enough.

The proofs in the main text for Theorems 2.1, 2.6 and 2.7 should either refer directly to the Appendix where they are proved, or be proved right there.

Regarding Theorem 2.8, the comment in the 'proof' space makes me think the Authors were not the first to prove it, in which case they should indicate it explicitly; otherwise it would be plagiarizing.

Currently Section 4 is quite lacking, including the aforementioned Table 1 which I cannot comprehend (by the way, it is missing a reasonable caption). 

A proposed method like this should be thoroughly tested, which in the current state of the paper it has not been. The methods the authors refer for comparison should include appropriate references.
Yes
No ethics review needed.
The authors mention in the Discussion that their method requires the boundary conditions of the PDE to satisfy a smoothness condition, specifically that f is in C^2, and that this requirement broadens the scope of their approach. However, it seems that C^2 smoothness could be more restrictive than a Lipschitz assumption. Could the authors clarify how they view this requirement as less restrictive? Additionally, could they discuss any potential limitations this might introduce for functions that are not in C^2?
A significant strength of this work is the innovative linking of the diffusion model to high-dimensional partial differential equations (PDEs), with Physics-Informed Neural Networks (PINNs) effectively employed to overcome the curse of dimensionality in solving these PDEs.
The authors propose the Feynman-Kac Operator Expectation Estimator (FKEE) to approximate the target distribution E[f(X)]. This estimator contains two parts: (1) A diffusion bridge model with parameters optimized to minimize the Wasserstein distance to the target distribution, and (2) a method based on the Feynman–Kac equation, formulated as a partial differential equation (PDE) and solved approximately using Physics-Informed Neural Networks (PINNs), which employ a least-squares approach. The experiments focus on approximating the partition function in a random graph model.
1. The Feynman–Kac model (Algorithm 2) with the PINN solver lacks a convergence or error estimate, which would be valuable for assessing its accuracy and reliability.

2. In the experiments, the authors claim that "using fewer points on the Markov chain achieves higher accuracy in approximating expectations." However, it is unclear if this result generalizes beyond the specific example provided, as it appears quite context-dependent.
Yes
No ethics review needed.
1. I don't know why and when MCMC is required to impose complex constraints on the distribution and performance function. Some references are suggested.

2. I don't see when MCMC is not the optimal decoding method. It appears to me that burn-in is not a significant limitation and only affects the performance on a negligible scale and can be easily fixed via a large stepsize in the beginning for warm-up.

3. Discussions on the limitations would be preferred.
the idea of using Feyman-Kac to approximate the expectation is interesting.
The authors proposed to leverage the Feynmann-Kac equation via Physically Informed Neural Networks (PINN) to approximate the target Mathematical Expectation efficiently and heuristically without causing a large variance.
The scalability of the algorithm w.r.t. dimension is not verified sufficiently. d=20 is too small. There are no real-world simulations.

The authors criticize the large variance issue by the MCMC method but fail to justify theoretically why the proposed method yields a lower variance. The empirical support is limited.

NIT: Theorem 2.1: the discretization error by Growall inequality is weak and exponentially dependent on time. Girsanov can be used to fix it.
Yes
No ethics review needed.
+ What are the limitations  of your methods regarding dimension? 

+ How does your algorithm compare with other neural ODE/SDE, diffusion models, bridges models in the literature?
Developing new sampling methods using ideas from generative model is an active area with lots of promising recent advances.
The paper presents two generative models to use in the context of sampling: a diffusion bridge with $W^2$-loss and algorithm based on solving the Feynmann-Kac PDE using PINNS.
+ There is no really new theoretical contributions in this paper. All the theorems mentioned in the paper are standard results. The diffusion bridge model has been used in various iterations in countless papers in the literature, for example by Doucet and collaborators.  

+ There is no indication that the algorithms presented here will scale up with dimension, especially if using the W^2 loss, so the claim that this improves on MCMC seem somewhat overblown.

+ Using a PINNs to solve the Feynmann-Kac equation is very unlikely to work in high dimension as PINNs are usually are no easy to to train.   

+ Lack of experiments on high-dimensional data sets.

+ No head-to-head comparisons with state of the art algorithms.
Yes
No ethics review needed.
Can the author disentangle their works from past literature on neural SDE and Feynman-Kac's use of averaging observables?

Can the authors provide evidence that their universal sampler outperforms the existing techniques, including those in Blechshmidt
et. al.(2021)? Can they provide some canonical well-excepted benchmark at which they excel over others?
The work addresses an important fundamental topic. It provides an interesting combination of DNN-based techniques.
The manuscript combines several DNN-based numerical techniques, associated with solving and learning SDEs and their PDE counterparts, with the aim of sampling from distributions and averaging over observables. Specifically, a type of neural-SDE is put forward and trained to converge to the desired distribution. Next averages of functions are estimated using the Feynman-Kac formula, wherein a PDE is derived from the learned SDE which “calculates” the averages for us given that one sets the boundary condition to be the function one wishes to average. This complex problem is solved using a PINN approach. Some numerical results on random graphs are reported in a table.

While the paper suggests an interesting technical path to explore in this important subdomain MC samplers— it is not clear from reading the manuscript how much of this is incremental work stringing together a series of known results or rather an original and meaningful step forward. While I am only a causal user of Monte-Carlo, my strong feeling is that the benchmarks shown are insufficient to prove that
this specific combination of techniques outperforms existing ones (including DNNs/PINN-based ones and more advanced MC techniques). Turning to the techniques themselves, it seems that neural-SDEs preceded the current work [Tzen, Ragidsky 2019], and that using Feynman-Kac formula with a combination of DNNs to estimate observables on an SDE has also been done before [Blechshmidt, Ernst 2021 and refs therein]. Reading into the latter work, it seems that DNNs have been used slightly differently than in the current work to solve the PDE associated with the SDE, but is this a conceptual change given the common use PINNs to solve PDE? Does it hold the key to any SOTA results? The related work section (which has been delegated to the appendix) and the general causal referencing to related works (such as Tzen et. al.) leave the non-expert reader with little understanding of the true novelty of the current results.

The presentation of the work also leaves a gap between conceptual claims and practical contributions. It also feels fragmented and the
common use of signposts and bold notation only worsens this in my mind. The conceptual claims, which are sometimes grand, are hard to substantiate. For instance, “Establishing a Link Between Sampling Methods and High-Dimensional Partial Differential Equations”, taken at face value, can hardly be attributed to the current work with all the knowledge on SDE, Fokker-Plank Equations, and Feynman-Kac formula. Also “Expanding the Scope of Expectation Estimators“, feels vague. Is there a current scope of expectation estimators? What results in the current work expand this scope in a way that others can't? Finally, in their introduction, the authors allude to the fact that the authors have an affirmative answer to the question “Is it possible to unify most existing MCMC algorithms into a cohesive framework to create a universal sampler for expectation estimation?“--- This is such a rich and complex problem that providing an affirmative answer would clearly violate various prevalent complexity theory assumptions. For instance, can the authors show that their sampler solves the Ising Spin-Glass problem? Can the authors even solve the much simpler case of the 2d Ising model at the phase transition and compute long-range observables? Does their technique outperform various ones used in physics to overcome sampling problems such as replica-exchange Monte Carlo?

The above issues, concerning the entanglement with previous works, evidence of going beyond SOTA, and its portrayed grand scope, prevent me from recommending it for publication in ICLR.
Relation to past literature is left vague. It does not show SOTA on a sufficient set of canonical problems with a suitable comparison to
other techniques. Its claims feel too broad and general. The writing style can be improved.