BAYESIAN EXPERIMENTAL DESIGN VIA CONTRASTIVE DIFFUSIONS
Bayesian Optimal Experimental Design;Conditional Diffusion Models;score based sampling;Bayesian Inverse Problems;Experimental Design;Sampling as Optimization
I thank the authors for answering some of my questions so quickly. Indeed, I was referring to LF-ACE from "A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments". I appreciate the authors taking the time to point out the gradient is different but they still miss a core of my question. If this isn't a lower bound on the information or an approximation, what is the EIG they are optimizing? I'm not talking about the gradient of the estimator but the estimator itself. Indeed, if it's not a lower bound by the InfoNCE using the method from Foster et al 2020, then what is it? 

As for my comment about a direct approximation of the EIG, the authors are correct that they do not state that explicitly in their work and I was wrong. I assumed it by extension from my previous question and the allusion in line 114. I see that the authors are relying heavily on the work of Goda et al. 2022 and Ao and Li, 2024. I'll note they also do not explicitly state which bound of the mutual information they use. Following equation 2 from that paper, 
$U(\lambda) = \mathbb{E}_{p(\theta|y, \lambda)}\left[\log l(y|\theta, \lambda)\right] - \mathbb{E}_{p(y|\lambda)}\left[\log p(y|\lambda)\right],$
 I can assume it's the InfoNCE lower bound since this the log ratio of the likelihood to marginal likeliihood. By extension, it can be shown the ACE gradient in equation 16 of Foster et al. 2020 is equivalent to the estimator in this paper since the L contrastive samples are used in a similar manner as the contrastive samples from the expected posterior in this paper. I'm dwelling on this because we know you can improve the information gain by increasing L but I'm encouraging the authors to think about how the expressiveness of the score function that approximates the expected posterior can be improved either by choice of importance sampling or neural network.

I'm also still wondering why the SNMC upper bound for their method is much worse than the others. It honestly doesn't matter in practice but it's a weird result to display.

I have more questions and comments after thinking this through and thanks to the authors clarification:
- What is the architecture of the diffusion model that's central to this paper? There's no mention and no code to reproduce the results. Appendix F has some details but I think more is needed to reproduce the results.
- How is the diffusion model pretrained since it's not being optimized in Algorithm 2?
- In line 365 the authors claim that running a conditional diffusion model is intractable yet that's exactly what this [paper](https://arxiv.org/abs/2209.14249) does so I don't think that method can be ruled out.
- I agree with the other reviewers you should state for the BOED community that you rely on explicit likelihoods. 

I apologize to the authors for being so pedenatic. I find this paper interesting and would like to understand it better with their help in explanations. Right now I'm left confused by the organization and some missing pieces that I think we can resolve.
Follow up to clarification about prior work
We thank the reviewer for this review and for acknowledging the "interesting contribution" and "great connection" made by our paper.
We are of course willing to clarify all points raised, but we are currently facing some difficulties in understanding some of the reviewer critiques. The reviewer claims we have "misunderstood the basis of BOED" and refer to a reference "Fotster et al 2019b" and to LF-ACE. We suspect a misspelling of "Foster et al 2019b", but such reference does not exist in our bibliography. Does LF-ACE instead refer to "Likelihood-free ACE" in section 3.4 of "Foster et al 2020- A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments" ?


LF-ACE, $I_{ACE}$, and InfoNCE have the same structure that leads to gradient estimations different from ours. Taking $I_{ACE}$ from eq. (11) of Foster et al 2020:

$$
I_{\text{ACE}}(\xi, \phi, L) 
= E_{p(y | \theta_0, \xi) p(\theta_0)} \left[ \log p(y | \theta_0, \xi) - \log \left(\frac{1}{L+1} \sum_{\ell=0}^{L} \frac{p(\theta_\ell)}{q_\phi(\theta_\ell | y)} p(y | \theta_\ell, \xi)\right) \right]
$$

As visible in eq. (19) of Foster et al 2020, its gradient $\nabla_\xi I_{ACE}$ differs from our gradient form: 

$$
\nabla_\xi EIG(\xi) = E_{p(y|\theta_0,\xi)p(\theta_0)} \left[ \nabla_\xi \log p(y | \theta_0, \xi) - E_{q(\theta' | y, \xi)} \left( \frac{p(\theta' | y, \xi)}{q(\theta' | y, \xi)} \nabla_\xi \log p(y | \theta', \xi) \right) \right]
$$


$I_{ACE}$ is both a lower bound of the EIG and  based on a variational approximation of the posterior, which is orthogonal to our approach. Also, we do not use a variational approximation of the posterior and instead use diffusion based samplers. 

For this reason, we believe the statement from the reviewer that our method "optimizes a lower bound of the mutual information by the InfoNCE bound" is not correct. Also, nowhere in the paper we write that our "approximation is exact". We focus mainly on the EIG gradient and not so much on the EIG itself. We do not emphasize the nature of the EIG approximation, which could be a lower bound, an upper bound or not a bound at all. 
Having said that we might have  "missed prior work that might better-inform our method" as the reviewer suggests. In such case, we are open to discussion but would appreciate specific references and detailed insights in what we missed.
Clarification request about prior work
Yes
No ethics review needed.
See the "Weaknesses" section.
The paper advances the field of BOED by making it more computationally efficient, whilst broadening the scope of problems that can be tackled by BOED by extending it to diffusion-based generative models. The idea of performing joint sampling and optimization using bi-level optimization seems novel to, although I am not very familiar with this literature. The experimental results are significant and demonstrate the efficacy of CoDiff compared to other baselines. The method is presented clearly along with discussions w.r.t. to the related works. Overall, this is a well-written and technically solid paper, and I am happy to recommend its acceptance.
This paper presents a new expression for the gradient of the expected information gain (EIG)---a central quantity in Bayesian optimal experimental design (BOED). Their method, named CoDiff, combines the two steps of maximizing the EIG and then sampling from the posterior in BOED into a single joint step by leveraging the sampling-as-optimization setting of e.g. Marion et al. (2024). Doing so results in a more efficient BOED method, as demonstrated by numerical experiments.
Some minor points:
* It would be nice to see some uncertainty bars around the results in Figure 3.
* Parts of Section 4 and 5 were a bit difficult for me to follow, but that could just be because I am not familiar with the relevant literature. However, if possible, it might be a good exercise for the authors to think about making these parts a bit more accessible to the BOED community who are not familiar with the sampling-as-optimization literature.
* It would have been nice to empirically test the performance of CoDiff as e.g. the dimensions of $\theta$ or the hyperparameters (if any) vary.
* Please include some limitations of CoDiff in the conclusion section.
Yes
No ethics review needed.
- Can the authors provide an intuition as to why the proposed expected posterior is preferable to the true expected posterior = prior? Is this purely for the sequential setting?
- how are the $\nu_i$ values determined in the expected posterior?
- what stop other methods from being used on the MNIST example? Is it just the computational cost from multiple nested loops and the sequential use case?
- rigorously cites and discusses related ideas
- merging multiple nested loops into a single loop is intuitive idea and has been applied in multiple separate fields (training a VAE is effectively the EM algorithm was simplified into a single step, in Bayesian optimzation few-shot Knowledge Gradient)
- the writing is clear and each idea is easy enough to follow.
This paper is a little outside of my expertise and I apologize in advance for any misunderstanding I may have and look forward to any correction from the authors. I have set my confidence to low accordingly.

The authors consider the problem of experimental design, given a statistical model of the form $P(y | \xi, \theta)$, we can choose values of $\xi$ and observe a corresponding $y$, and the goal is to infer $\theta$. In the first example in the paper, $\xi$ are locations on a 2D plane, $y$ are the measurements at the $\xi$ locations, and $\theta$ represents the source locations of contamination in the plane.

Assuming we have chosen the $\xi$ values, we can use the $p(y|\xi)$ to simulate $y_i$ values, one hallucinated rollout into the future, then infer the hallucinated posterior $p(\theta'|y_i, \xi)$ and measure the posterior distribution's entropy, $H(p(\theta'|y_i,\xi))$, typically requiring Monte-Carlo over the $\theta'$. We may repeat this process many many times for $i=1,...,N$, and compute the average posterior entropy, wrapping the inner Monte-Carlo over $\theta'$ in another outer Monte-Carlo over $y$. This average posterior entropy is the Mutual Information (MI) between $y$ and $\theta$ up to an additive constant (that is the prior entropy $H(p(\theta))$). This MI quantifies the benefit of sampling $y$ at $\xi$, we can therefore change $\xi$ in a way to maximise MI and this is a standard method for experimental design known as Expected Information Gain, or EIG.

However as evaluating EIG requires performing nested Monte-Carlo, and optimizing $\xi$ requires the gradient of EIG, much work has focused on various methods to construct efficient EIG gradient estimators. In past works and this works, the second inner integration over $\theta'$ is performed with MCMC or by Monte Carlo with importance sampling using a proposal $q(\theta'|y_i)$ that must be updated for every outer MC iteration $i$.

This work proposes to use a single proposal distribution across all outer MC iterations $i$ called the "expected posterior" which is a geometric mixture of all the hallucinated posteriors
$$
q(\theta') \propto \prod_i p(\theta|y_i, \xi)^{\nu_i}
$$

The paper proposes to Langevin dynamics to generate samples for the outer Monte Carlo $\\{y_i, \theta_i\\}$ from the joint distribution. As these samples are updated in a iterative manner, and the goal is to iteratively optimize $\xi$, these nested iterative procedures can be merged and updated together, if $\xi$ only incrementally moves, it is wasteful ti discard all the old samples $\\{y_i, \theta_i\\}$ but keep incrementing them so they're up-to-date.

Finally the proposed method is applied to two sequential design applications, the first is the source location example in which the $p(\theta)$, $p(y|\theta, \xi)$ have known analytical forms and the proposed method outperforms baselines. The second is where $p(\theta)$ cannot be evaluated but can be sampled, specifically a diffusion model trained on MNIST images, $\theta$ is a full image,  $\xi$ are pixel locations, and $y$ are the 7 X 7 patch of pixel values around $\xi$. The goal is to infer the full image by cherry-picking image patches.
### Technical Comments
- __expected posterior__ the paper states that they aim to find one proposal distribution that covers all possible future posteriors and proposes a distribution that is a geometric mixing of hallucinated posteriors $q(\theta) \propto \prod_i p(\theta|y_i, \xi)^{\nu_i}$. The prior is the true expected posterior $p(\theta) = \mathbb{E}_Y[p(\theta|y)]$, exactly the average density of all possible posterior densities.  Why not use the prior $p(\theta)$ as the one global proposal distribution for integration over all the hallucinated posteriors? In the sequential case (introduced at the end of the paper)

### Writing Comments
- __Writing Density__ while the writing is thankfully clear, the paper cites and quickly introduces many ideas from prior works in quick succession making the paper very hard work to read for me, a lot of context switching making the paper very tiring. I would suggest reducing the details of referenced works, and providing higher level descriptions where possible and elaborating more on the novel parts of the proposed method.
Yes
No ethics review needed.
My questions are contained in the Weaknesses section.
- The authors make a great connection to optimization by sampling via diffusion models. 
- Thorough gradient analysis.
- The connection to inverse problems e.g. inpainting in MNIST images as a BOED problem is an interesting new task in the BOED setting and could see BOED objectives (really mutual information objectives) gain further adoption in generative modeling.
This paper makes an interesting connection between Bayesian optimal experimental design (BOED) and optimization by sampling using a diffusion model. While an interesting connection that may prove fruitful with some refinement, the authors seem to misunderstand the basis of BOED and maximization of a lower bound of mutual information using contrastive learning, as well as miss prior work that might better-inform this method. This disconnect, as well as confusing notation regrading how the diffusion model is used in multi-round BOED and lackluster empirical evaluation, makes me think this work is too early and could benefit from refinement in writing and additional experimental evaluations of the method.
While the authors make a great connection to optimization by sampling, theoretical analysis of the convergence of the desired goal distribution is missing. This would e.g. especially be helpful in the multi-step BOED setting to show how to improve bounds of the mutual information. I list out my critiques and questions below:

- The authors miss that they are also optimizing a lower bound of the mutual information by the InfoNCE bound, thus their comments that their approximation is exact is incorrect.
- The definition of information in Equation 2 does not make sense. The KL divergence is written in a non-standard way - do you mean $E_{p(y|\xi)}KL(p(\theta|y, \xi) || p(\theta))$?
- The authors miss the connection to Fotster et al. 2019b of LF-ACE that uses a posterior estimate in the InfoNCE bound to refine samples. e.g. in equation 5. 
- The notation for the expected posterior in equation 9, it's not clear what is the proposal distribution $q(\theta)$ in the paper. Are you using likelihood evaluations? Also, how are you choosing the weighting of $\nu_i$?
- I would prefer to see the notation for iterative sampling operators similar to Marion et al. 2024, where the Y, and $\Theta$ are variables of the operator instead of in the superscript. I think that would make the optimization problem more clear. 
- After equation 15, are you using two generative models to define the observed distribution $y$ and the parameter distribution $\theta$?
- in terms of the location finding experiment, usually we would expect to see the designs and final posterior distribution. the way it's portrayed in the paper is confusing. I'm not sure which samples are shown, which design round the designs come from, and why the seeming optimal design isn't on the ground truth values. It looks like the chosen design is quite far from the true theta_0.
- While the EIG seems to improve over others in Figure 3, the upper bound is _much_ worse. Why is that? 
- The MNIST experiment is interesting but lacks quantitative evaluation. Maybe including a classifier and using the number of successfully classified images would help? Also, there seems to be opportunity to change the design size to see how that influences outcomes.

In summary, I think the notation is a little bloated. The authors tend to repeat themselves in the text and could be more concise. Concision would help make space for exposition of more salient points that would improve this paper. Additionally, it would help the authors to explicitly lay out what are the likelihood, posterior, and priors they use in experiments. Also, it would be nice to see calibration of the posterior that results of the final design round using e.g. the l-C2ST method or other simulation based calibration methods. Most applications of amortized posteriors using BOED will be interested in this, but it can be computationally expensive, which is why I think just on the final round is sufficient. This is an interesting method that I think needs cleaner exposition.
The paper introduces an efficient BOED method leveraging diffusion-based samplers and bi-level optimization ideas to jointly sample the introduced expected posterior and maximize Expected Information Gain, enabling larger-scale applications.
@inproceedings{
anonymous2024bayesian,
title={{BAYESIAN} {EXPERIMENTAL} {DESIGN} {VIA} {CONTRASTIVE} {DIFFUSIONS}},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=h8yg0hT96f},
note={under review}
}
Bayesian Optimal Experimental Design (BOED) is a powerful tool to reduce the cost of running a sequence of experiments.
When based on the Expected Information Gain (EIG), design optimization corresponds to the maximization of some intractable expected  *contrast* between prior and posterior distributions.
Scaling this maximization to high dimensional and complex settings has been an issue due to BOED inherent computational complexity.
In this work, we introduce an *expected posterior* distribution with cost-effective sampling properties and provide a tractable access to the EIG contrast maximization via a new EIG gradient expression. Diffusion-based samplers are used to compute the dynamics of the expected posterior and ideas from bi-level optimization are leveraged to derive an efficient joint sampling-optimization loop, without resorting to lower bound approximations of the EIG. The resulting efficiency gain allows to extend BOED to the well-tested generative capabilities of diffusion models. 
By incorporating generative models into the BOED framework, we expand its scope and its use in scenarios that were previously impractical. Numerical experiments and comparison with state-of-the-art methods show the potential of the approach.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Bayesian Optimal Experimental Design Conditional Diffusion Models score based sampling Bayesian Inverse Problems Experimental Design Sampling as Optimization
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/c361c6f3cfbfead127cece423f11e2b9b4abe8d3.pdf
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/e7b6dbe2dc8d421cd140101960fdb9f2d64335d6.zip
BAYESIAN EXPERIMENTAL DESIGN VIA CONTRASTIVE DIFFUSIONS
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission