FROM LOW TO HIGH-VALUE DESIGNS: OFFLINE OPTIMIZATION VIA GENERALIZED DIFFUSION
Diffusion Model;Probabilistic Method
Yes
No ethics review needed.
- Why do you set the value of zeta to t/T? and delta_t to zeta_t(1-zeta_t)?

### Suggestions:

- Could you consider revising your abstract? The goal of the paper was not immediately clear.
- Please consider a rewrite of section 3.1. Start with what the optimization function is. Then follow through with the derivations on how it was obtained. Please mention how this is used in an algorithm.
- Happy to raise my score given some changes to the manuscript.

### Typos
- Line 233: "as follows"
- The use of diffusion in design tasks is quite novel. 
- The results are strong. But note that the best of 128 samples is reported in Table 1. The algorithm itself may not have access to "best score". So while the algorithm reaches good results in best case, median or avg may have to be considered.
This is a high-potential paper which could be much better written. I suggest a full review of the paper for readability. For example, the second line of the abstract is 4 lines long.  The goal of the paper is not clear from reading the introduction. 

The paper seeks to learn surrogate parameterized functions which learn the data distribution. Using these surrogates, the data is augmented, and this augmented data can then be used to train a model. The issue with such augmentation is that both the (x,y) pairs must be learnt, and predicting the x may lead to OOD generations. This is usually solved through inverse mappings, or search. This paper proposes ideas from Brownian bridge diffusion, which was previously proposed in computer vision to extend to tasks in the Design Bench.
- Why use the brownian bridge diffusion process? How do other diffusion processes perform?
- Section 3.1 what is the stochastic translation operator? why use terms before defining them?
- The paper has spent a lot of space on Section 3.1 which comes without explanation of what the goal is. For example what is g in this setting? Are these the equivalents of the noise estimates in DDPM? What is the reason for the specific form in equation 13? Please rewrite thsi section with what you are attempting in this section. While reading this section, it is extremely unclear what the mathematics is leading towards, and hence why a stochastic translation is required.
- There is no algorithm section translating the mathematics of Section 3 into the experiments of Section 4. I do not see what was the optimization implemented, or how it was obtained.
- The paper does not report running times of the algorithms. Given diffusion can be significantly costlier it is worth reporting the wall-clock time.
- The paper reports max score in Table 1, but without external evaluation, it is not clear that the algorithm may know what is best. Hence this is not the expected performance. Can  you also show the mean score?
Yes
No ethics review needed.
In the paper, notation $\omega$ is used in multiple places. Is it a random variable indicating the randomness of the underlying function and offline data? In Eq (12) and follows, is it $\omega^*$ or $\omega$?
The problem of offline optimization is important and has many potentials. 

The experiments are presented clearly and the proposed method indeed has improvements compared to baselines.
This paper considers the offline optimization problem, where offline (x, y) pairs are given, and the goal is to return samples x with better value. The paper proposes a distribution shifting approach by first learning the underlying function via a mean estimate of the Gaussian process under difference kernel functions and creating two sets of samples $X^+$ and $X^-$ by taking gradient ascent and gradient descent based on the estimated function starting from the offline samples. Then a diffusion model is learned via translating the data from $X^-$ to $X^+$ via assumed conditional Gaussian distribution. Experiments are conducted on the variance of benchmarks and the proposed method is compared to be comparable/superior to some offline linear optimization baselines.
The originality of viewing the offline optimization as a distribution shift is not novel. See [a]. 

The derivations are not correct. The proposed method is not theoretically justified, merely a theoretically motivated heuristic. For example, the Assumption of Eq (13) cannot be justified given Eq (12), thus giving no evidence for Eq (14) as combining both equations. Thus the overall learning procedure lacks of justifications. Take another example of Eq (3), the optimal solution x^* is a function of the unknown function g, and thus g(x^*) conditioned on the offline data is not Gaussian. It is the maximum of several Gaussian since $x^*$ is also a random variable. 

The diffusion approach generates data outside of the distribution of the samples (outside of the distribution coverage) thus leading to critical generalization issues. This issue is not sufficiently addressed. 

Overall, it is a work based on a wrong assumption/mathematics though has engineering improvement. 

[a] Zhang, Q., Zhou, R., Shen, Y. and Liu, T., 2024. From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimization. arXiv preprint arXiv:2401.02019.
Yes
No ethics review needed.
NA
1. The new perspective of viewing the optimization problem as a diffusion translation problem is quite novel and intuitive.

2. The method proposed in this paper is direct and clear: It uses synthetic data from fitted gaussian process for data augmentation and then employs a diffusion model to learn the distributional translation from low-value to high-value regions.

3. The presentation of this paper is clear and well-structured.

4. The experiments are comprehensive.
This paper presents a new perspective on offline optimization, framing the optimization task as a distributional translation from low-value points to high-value points. It uses diffusion models to learn this translation. The paper shows the effectiveness of this new perspective through theoretical analysis and experimental validation.
The method needs to fit multiple Gaussian processes to generate synthetic data, which might have high time complexity in high-dimensional cases. Besides, the reverse diffusion process also requires multiple denoising steps, which also increases the time complexity.
We view offline optimization from the new lens of a distributional translation task which can be modeled with a generalized Brownian Bridge Diffusion process mapping between the low-value and high-value input distributions
@inproceedings{
anonymous2024from,
title={{FROM} {LOW} {TO} {HIGH}-{VALUE} {DESIGNS}: {OFFLINE} {OPTIMIZATION} {VIA} {GENERALIZED} {DIFFUSION}},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=K9Elg2JrvY},
note={under review}
}
This paper presents a new perspective on offline optimization. Instead of viewing it as a surrogate or inverse modeling task -- mapping either from an input design to its corresponding performance or from a desired performance to potential input candidates -- we approach offline optimization as a distributional translation task that transforms an implicit distribution of low-value inputs (i.e., the offline data) into a (better) distribution of high-value inputs (i.e., the solution candidates). This avoids explicitly modeling the target function, which is ultimately constrained by the limited amount of offline data. In contrast, our view of offline optimization as a distributional translation task is substantiated through a generalized Brownian bridge diffusion process mapping between two implicit data distributions, which can be more reliably learned using additional low- and high-value inputs drawn from synthetic functions similar to the target function. This is enabled by fitting multiple Gaussian processes with different parameterizations to the offline data and using them as functional posterior to generate artificial functions similar to the target function. Our experiments show that this approach is consistently more effective than previous methods, establishing a new state-of-the-art performance.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Diffusion Model Probabilistic Method
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/0a7a529b3a881fe7cd6f4a292e5b3bf6cfa3eb00.pdf
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
FROM LOW TO HIGH-VALUE DESIGNS: OFFLINE OPTIMIZATION VIA GENERALIZED DIFFUSION
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission