Gradient-Free Generation for Hard-Constrained Systems
Flow Matching;Generative Model;Constrained Generation;Partial Differential Equations;Conservation Laws
Yes
No ethics review needed.
1. How does proposed ECI compared with FFM without ECI in experiments? It would better show the improvement from ECI in sampling. 
2. In section 3.3, the authors mention resampling noise during sampling. Can the authors explain why this helps better generative results?
3. The model assumes a perfectly trained FFM, however there is inevitable error in the trained generative model. How does such error affect the performance? Will such error break the constraint in sampling?
4. Empirical results are on low-resolution problems. Can the authors comment on how ECI work at larger scale?
1. The work is well-motivated, it's important in scientific applications to have constraint generation
2. The proposed method is effective and does have constrained generation. Also, the method is zero-shot which is a big benefit for constraint sampling method. Although it's based on functional FM not FM in general
3. The experimental results show advantages over other baseline models.
In this work, the authors propose ECI for gradient-free generation for constraint systems. ECI is zero-shot and doesn't need tuning the model. It works through alternating between extrapolation (E), correction (C), and interpolation (I) in each sample step. The empirical results show the model achieves competitive performance across various PDE systems compared with baseline models.
1. The writing is also a bit confusing. Actually comments from reviewer KM5e help me better understand the algorithm 2. 
2. The authors are recommended to better formulate the contribution of this work. In particular, the model works in functional space and applies projection to constraint spaces to guarantee hard-constraint met. 
3. The authors mention supply chain optimization in abstract and introduction as a hard-constrained system. However, it lacks experiments on such problems. It would be better to showcase some applications beyond PDE learning (which I believe is broad).
Yes
No ethics review needed.
1. This is the most critical question. How does ECI guarantee that the achieved solution satisfies the PDE system? In computer vision tasks where differences from ground truth images are permissible.  In this work, boundary conditions are guaranteed, but how does ECI ensure that the generated data truly represents solutions to the PDE? More specifically, why the linear interpolation between two PDE solutions is still a valid PDE solution?

2. Can you elaborate more on the stochasticity of generated solutions, as is discussed in Sec 3.3. Does more stocahsticity mean the generated distribution is distribution-wise closer to the uniform distribution over the space of constrained PDE solutions?
3. For the extrapolation step, will the algorithm performance be better if more a refined scheme is used? For example, mutiple step extrapolation when time $t$ is far from $1$ and one step when $t$ is close to $1$. From my understanding, the quality of predicted $\hat u_1$ is crucial to the algorithm performance. An additional ablation study could be performed here.
4. Can you comment on the computational efficiency of ECI compared with existing approaches?

The current score assumes a positive answer to the first question. My score will be adjusted accordingly based on the authors' response.
1. The paper is easy to follow with high readability. The problem setting is well motivated and important.
2. The paper provides quite comphrenseive numerical results with comparison to relevant benchmarks. The extension to regression setting is also quite impressive.
3. The paper also provides a quite detailed ablation study on the choice of algorithm hyperparameters.
The paper introduces ECI sampling, a framework that adapts pre-trained, unconstrained generative models to exactly satisfy hard constraints in a zero-shot manner without requiring gradient computations or fine-tuning. This framework alternates between extrapolation, correction, and interpolation stages during each iterative sampling step to integrate constraint information while preserving the validity of generated outputs. Empirical results demonstrate that ECI sampling strictly adheres to physical constraints across various PDE systems and outperforms baseline approaches in zero-shot constrained generative and regression tasks.
*1. Clarification of Problem Setup*

The problem setup is not sufficiently explained, which may lead to confusion. Although the authors repeatedly emphasize that ECI is intended for the generative modeling of constrained PDE solutions, it remains unclear what distribution of constrained solutions ECI aims to recover. Specifically, is it targeting a uniform distribution over the space of constrained solutions $\mathcal{U}_{|\mathcal{G}}$? The authors should invest more effort in formally clarifying the problem setup.

*2. Discussion of ECI’s Advantages and Comparisons*

The paper appears to lack a section that discusses the benefits and potential sources of ECI’s superior performance, as well as comparisons with other existing approaches mentioned in the work. Including such a discussion would help readers better understand both the relevant literature and the proposed method.

*3.Validation of PDE System Satisfaction*

While ECI ensures the exact satisfaction of the constrained operator by applying a correction operator after each one-step extrapolation, it is unclear why the solution obtained at time 0 still satisfies the PDE system $\mathcal{F}_{\phi} u(x) = 0$. In Algorithm 2, the next-step solution is generated by forward noising starting from the predicted solution at time 1. Does this noising process preserve the satisfaction of the PDE system? Additionally, is the linear interpolation of two PDE solutions a valid PDE solution, implying that the PDE system is linear? Further explanations are needed to clarify these points.

*4.Missing References on Gradient-Free Guidance*

Some references on gradient-free guidance are missing, such as [1] and [2]. It appears that the proposed method is related to [2], involving functional FM and additional projection steps. Providing more clarification on these connections would be appreciated.

[1]Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with nondifferentiable rule guided diffusion. arXiv preprint arXiv:2402.14285, 2024.

[2]Chung, Hyungjin, et al. "Diffusion posterior sampling for general noisy inverse problems." arXiv preprint arXiv:2209.14687 (2022).
We thank your recognition of our work's **extensive evaluations** and your constructive suggestions to improve the presentation of our work. We apologize for some not-well-explained notations with our assumption that all reviewers were familiar with generative models, especially flow matching. We have included a more detailed mathematical introduction to flow matching in the common rebuttal session, and we will further elaborate on some additional details to address your questions and concerns.

## Q1 Notations

We first clarify some notations in addition to Section 3.1. We use $u:\mathcal{X}\subseteq\mathbb{R}^d\to \mathbb{R}$ to denote a continuous function as the solution. The subscript always indicates the time step in the flow. We follow the flow matching convention such that $t=0$ corresponds to the noise and $t=1$ corresponds to the target (data). As the space of continuous functions has an infinite dimensionality, we follow FFM [1] to adopt measure-theoretical terms of $\mu_0,\mu_1$ as probability measures over the continuous functions, instead of using the common $p_0,p_1$ in computer vision domains. All $p,q$ in our work are probability densities defined as the  Radon–Nikodym derivative (see the FFM paper for technical details). Specifically, at each time step $t$, the noised data $u_t:=\psi_t(u_0|u_1)$​​ can be obtained with *interpolation*. We noted that **all of the above definitions are standard notations in flow matching context**.

We do agree Equation 2 is less well-defined as we postponed the definitions in the bulletin list later. We will **modify the order in our revised manuscript** to first introduce and define the notations to avoid potential confusion. We will also follow your suggestion to move Proposition 1 to the main text to make it more understandable.

## Q2 Algorithm

Your understanding of our Algorithm 2, the core algorithm for ECI sampling, is definitely correct. We originally put it in a more concise way mainly because **all notations are quite standard in diffusion or flow-based generative models**. For example, [2, 3] also adopts an iterative formulation for guiding diffusion models with a double loop. We will consider reformatting the algorithm to include more details following your outline.

Still, we would like to mention that both extrapolation and interpolation stages for flow matching have clear definitions with respect to the probability path induced by the flow. Specifically, the linear interpolation corresponds to the *OT-path* (OT for optimal transport) in flow matching literature [4] with theoretical benefits of optimal transport. It is **not our whimsical assumption but theoretically a good option** in choosing the probability paths. 

Also, due to the iterative nature of flow sampling, ECI sampling needs to advance the solver for the flow ODE (see common rebuttal session). Therefore, the last step of advancing the solver to a large $t'=t+1/N$​ is also necessary.

## Q3 Other Remarks

- Regarding $\mathcal{F}_\phi$. Your understanding is correct. We will clarify it.
- Regarding the domain of $x$. We assume the normal domain of $\mathcal{X}$ for $\mathcal{F}_\phi$.
- We used the notation $\mathcal{U_{|F}}$ to indicate the solution set is *restricted* by the constrained to a narrower subset.
- We used $:=$ for definition (of new quantities). For the push-forward, we use the standard notation of $(\psi_t)_*$​, e.g., the second part of Section 3.1.
- For the correction algorithms in Algorithm 3&4, they serve as concrete examples of the correction $C(u_1,\mathcal{G})$ in Equation 3&4 and Algorithm 2.

We also thank your other suggestions regarding the notations and formatting for better conciseness and understandability. We will add more clarifications on notations, more mathematical backgrounds, and better presentations in our revised manuscript to make our work more accessible to a larger audience.



[1] Kerrigan, Gavin, Giosue Migliorini, and Padhraic Smyth. "Functional flow matching." *arXiv preprint arXiv:2305.17209* (2023).

[2] Lugmayr, Andreas, et al. "Repaint: Inpainting using denoising diffusion probabilistic models." *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022.

[3] Kawar, Bahjat, et al. "Denoising diffusion restoration models." *Advances in Neural Information Processing Systems* 35 (2022): 23593-23606.

[4] Lipman, Yaron, et al. "Flow matching for generative modeling." *arXiv preprint arXiv:2210.02747* (2022).
Rebuttal to 11X9
[1] Lugmayr, Andreas, et al. "Repaint: Inpainting using denoising diffusion probabilistic models." *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022.

[2] Kawar, Bahjat, et al. "Denoising diffusion restoration models." *Advances in Neural Information Processing Systems* 35 (2022): 23593-23606.

[3] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." *arXiv preprint arXiv:2010.08895* (2020).

[4] Li, Zongyi, et al. "Anima Anandkumar, Physics-informed neural operator for learning partial differential equations." *arXiv preprint arXiv:2111.03794* (2021).

[5] Herde, Maximilian, et al. "Poseidon: Efficient Foundation Models for PDEs." *arXiv preprint arXiv:2405.19101* (2024).

[6] Sun, Jingmin, et al. "Towards a Foundation Model for Partial Differential Equation: Multi-Operator Learning and Extrapolation." *arXiv preprint arXiv:2404.12355* (2024).

[7] Huang, Jiahe, et al. "DiffusionPDE: Generative PDE-solving under partial observation." *arXiv preprint arXiv:2406.17763* (2024).

[8] Négiar, Geoffrey, Michael W. Mahoney, and Aditi S. Krishnapriyan. "Learning differentiable solvers for systems with hard constraints." *arXiv preprint arXiv:2207.08675* (2022).
Rebuttal to KM5e (Reference)
We thank your recognition of our work's **comprehensive experimental results** and **concise formulation**. We are happy to address your questions and concerns. Still, we noted the potential misinterpretation of our proposed approach as a regression model but not as a generative model. See the common rebuttal for more details, clarifications, and mathematical backgrounds.

## Q1 Results in Figure 6-8

Figure 6-8 demonstrates the generation statistics of point-wise mean and standard deviation, together with the ground truth statistics. Similar to Figure 1&2, a closer resemblance to the ground truth statistics indicates a better generation result. It is worth noting, though, that MMSE and SMSE only reflect the first two moments of the distribution but not higher-order information. Therefore, FPD scores should be preferred, in which the pre-trained PDE foundation model Poseidon is used to extract the hidden representations (see Appendix D.1). We have included the detailed experimental setups in Appendix B and we will add more explanations for Figure 6-8.

## Q2 Experiments with different IC/BC

Thanks for the insightful suggestion. We are running experiments on the Stokes dataset with different IC/BC values to further demonstrate the generalizability and effectiveness of our approach. We will update the results once they are readily available. 

## Q3 Overall framework

The overall framework of our ECI sampling can be understood as a modification to the vanilla flow sampling process in Algorithm 1. With minimum modifications, we demonstrate the ECI sampling in Algorithm 2. Previous work in guiding diffusion models also adopts an iterative formulation [1, 2], which is in fact **standard in the context of diffusion or flow-based generation models**. We understand that the reviewer may not be familiar with the generative models and we have provided more details in the common rebuttal session.

Regarding the datasets used in our work, we noted all datasets and their spatiotemporal resolution setups are directly **taken from previous papers** (see Appendix B for reference of the datasets in previous papers). Therefore, we believe such an experimental setup is **standard, consistent, and comparable with existing works**. To the best of our knowledge, **4D datasets in PDE systems do not exist so far**. It is also a **standard practice** to test on 2D and 3D datasets in the PDE domain, e.g., neural operator learning [3, 4], foundation models [5, 6], and existing constrained generation [7, 8]. Therefore, we believe we have already provided enough evidence of the superior performance of our proposed method.

We already included the dataset descriptions, specifications, and task formulation in Table 2&7, Appendix B&D.2, for which our ECI sampling serves as a unified constrained generation framework.

## Q4 Improvement in the whole domain

We note that our ECI sampling was designed to offer **iterative control** in each flow sampling step to mitigate the artifacts between the constrained and unconstrained regions. Indeed, our generation result demonstrated **remarkable consistency in the whole spatiotemporal domain**, with hardly noticeable artifacts around the boundary in Figure 1, 6, 7, and 8. This is in sharp contrast with the naive projection algorithm like ProbConserv, which can be only applied to the final prediction without fine-grained control over each sampling step. In the close-up demonstration in Figure 5, noticeable artifacts can be found for ProbConserv and other gradient-based approaches.

The iterative control of ECI sampling that gradually pushes the intermediate noised data into generations with hard constraints can be further demonstrated in Figure 4. Note how the generation trajectory gradually exhibits more consistency between the constrained and unconstrained regions.

## Q5 Zero-shot effectiveness

We emphasize that **all experiments are carried out in a zero-shot manner** except for the conditional FFM baseline. Specifically, an unconstrained FFM was first trained as the generative prior. Different approaches including our ECI sampling were then applied in a zero-shot manner (see Appendix C&D.2). In this way, our framework provides a **unified approach for different constraints** without finetuning or conditional training (e.g., IC and BC in the Stokes problem). The generative evaluation metrics also demonstrated the state-of-the-art performance across all zero-shot models (conditional FFM is not zero-shot and should be viewed as an upper bound, as we have discussed in Section 4.1).
Rebuttal to KM5e
Dear Reviewers,

We sincerely appreciate your reviews that help make our work more concrete and clear. We thank you for pointing out the strengths in our work. Sadly, we note a common **misunderstanding and misinterpretation among all the reviewers** and we would first like to clarify this. 

### **Our proposed ECI sampling framework is NOT a regression model** (e.g., FNO [1]), **nor a plugin for existing regression models** (e.g., BOON [2]). **Instead, ECI sampling is a generative framework for functional data based on flow matching.**

We believe all reviewers have misinterpreted this setting and have asked for a comparison with regression models or using regression metrics, which are not applicable to our problem setup. We do apologize for our assumption that this paper's audience is familiar enough with generative models, especially flow matching. We now add a more detailed introduction to flow matching here to make our paper more accessible to the audience in the SciML domains.

## Flow Matching

Flow matching [3] is a generative framework built on continuous normalizing flows. Flow matching tries to learn the time-dependent *vector field* $v_t:\mathbb{R}^d\times [0,1]\to\mathbb{R}^d$ that defines a continuous time-dependent diffeomorphism called the flow $\psi_t:\mathbb{R}^d\times [0,1]\to\mathbb{R}^d$ via the following *flow ODE*:
$$
\frac{\partial}{\partial t}\psi_t(x_0)=v_t(\psi_t(x_0)),\quad x_0\sim p_0(x)
$$
The flow induces a probability path with the push-forward $p_t=(\psi_t)_*p_0$ for generative modeling. [3] demonstrated the conditional vector field $u_t(x|x_1)$ can be calculated simulation-free while sharing the same gradient with the flow matching objective. In short, flow matching models try to learn the conditional vector field which defines the dynamics of the flow and the probability path.

[3] further demonstrated the conditional probability path can be manually defined with linear interpolation between the noise and target (the *OT-path*). [4] further extended flow matching to Riemannian manifolds where the interpolation has a more intuitive interpretation of geodesic interpolation. Either way, it is clear that **in all flow-matching literature, extrapolation and interpolation have well-defined meanings with respect to the probability path induced by the flow**. 

Based on the above observation, we can clearly summarize the difference between regression models (neural operators) and our generative framework as follows:

- ECI sampling, as a flow-based generative framework, can **output a distribution of solutions**. In contrast, any regression model (neural operator) can only output one deterministic prediction.
- As a special case of the flow matching model, ECI sampling is **an iterative process that gradually *denoises*** the current noised samples into meaningful ones by solving the flow ODE above using the Euler method or any ODE solvers. See Algorithm 1 and Figure 4 for demonstration. In contrast, neural operators directly output the final prediction.
- The **learning objective of flow matching is completely different** from neural operators. The former tries to minimize the vector field discrepancy whereas the latter directly predicts the final outcome.
- The generative nature requires **generative evaluation metrics instead of regression metrics**. As both the output and the ground truth are a set of solutions instead of a single solution, distributional properties should be compared as generative metrics.

We further noted that the **iterative nature** of diffusion and flow-based models makes it **non-trivial to guide**, as intermediate steps are noises instead of final predictions. Indeed, we have included existing constrained generation and inverse problem works in computer vision in the related work. However, the continuous nature of flow matching makes their approaches not directly applicable.

We understand the reviewers may not be familiar with the generative domains. As our work lies in the intersection between scientific ML and generative models, we urge the reviewers to **take the contributions in generative modeling of our work — which is our major contribution — into consideration** to give a more comprehensive evaluation of this work.

Warm regards,

Authors.

[1] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." *arXiv preprint arXiv:2010.08895* (2020).

[2] Saad, Nadim, et al. "Guiding continuous operator learning through physics-based boundary constraints." *arXiv preprint arXiv:2212.07477* (2022).

[3] Lipman, Yaron, et al. "Flow matching for generative modeling." *arXiv preprint arXiv:2210.02747* (2022).

[4] Chen, Ricky TQ, and Yaron Lipman. "Riemannian flow matching on general geometries." *arXiv preprint arXiv:2302.03660* (2023).
Rebuttal to all reviewers
## Q8 Generation Performance

We have noted the **misunderstanding of our contributions** of ECI sampling, especially in terms of the correction stage in Q5 & Q6, and we have also noted the reviewer's **concerns for poor performance were ungrounded** in Q2. Indeed, across all 2D and 3D generative datasets, ECI sampling achieves state-of-the-art performance on most evaluation metrics. It is unclear to us **why the reviewer emphasized solely the MMSE for the NS equation but deliberately ignored all of the other metrics and datasets** in which our ECI-sampling achieved the best results.



# Other Questions

## Q9 Conservation Laws

It is always possible to apply alternative correction algorithms for conversation laws, but **unnecessary**, as 1) this correction stage is not our contribution; and 2) even the simple projection algorithm works pretty well with our iterative framework.

## Q10 Higher Dimensional Data

We reiterate that the correction stage is the least important stage in our algorithm, as even a simple projection algorithm can lead to decent results. Therefore, we believe our ECI sampling can be scaled to higher dimensional data while preserving its competitive performance. However, as we have mentioned in Q2, such data are, if not none, scarce for training a generative model. We believe our existing experiments across various 2D and 3D PDE systems already demonstrated the superior performance of ECI sampling.

## Q11 Frechet Poseidon Distance

As we have mentioned at the very beginning of our rebuttal, our ECI sampling is still a **generative framework** for which generative evaluation metrics for distributional properties should be used instead of the normal regression metrics. In this sense, **the ground truth is a set of solutions** with no one-to-one correspondence, making it impossible to calculate the regression metrics.

In contrast, inspired by the *Frechet inception distance* (FID), a widely used metric in image generation domains, we proposed to use the foundation model Poseidon to give a **more comprehensive evaluation of the distributional property**, in which our model excels over baselines. See Appendix D.1 for details.

[1] Saad, Nadim, et al. "Guiding continuous operator learning through physics-based boundary constraints." *arXiv preprint arXiv:2212.07477* (2022).

[2] Négiar, Geoffrey, Michael W. Mahoney, and Aditi S. Krishnapriyan. "Learning differentiable solvers for systems with hard constraints." *arXiv preprint arXiv:2207.08675* (2022).

[3] Li, Zongyi, et al. "Anima Anandkumar, Physics-informed neural operator for learning partial differential equations." *arXiv preprint arXiv:2111.03794* (2021).

[4] Mouli, S. Chandra, et al. "Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs." *arXiv preprint arXiv:2403.10642* (2024).

[5] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." *arXiv preprint arXiv:2010.08895* (2020).

[6] Li, Zongyi, et al. "Anima Anandkumar, Physics-informed neural operator for learning partial differential equations." *arXiv preprint arXiv:2111.03794* (2021).

[7] Herde, Maximilian, et al. "Poseidon: Efficient Foundation Models for PDEs." *arXiv preprint arXiv:2405.19101* (2024).

[8] Sun, Jingmin, et al. "Towards a Foundation Model for Partial Differential Equation: Multi-Operator Learning and Extrapolation." *arXiv preprint arXiv:2404.12355* (2024).

[9] Huang, Jiahe, et al. "DiffusionPDE: Generative PDE-solving under partial observation." *arXiv preprint arXiv:2406.17763* (2024).

[10] Kerrigan, Gavin, Giosue Migliorini, and Padhraic Smyth. "Functional flow matching." *arXiv preprint arXiv:2305.17209* (2023).

[11] Lugmayr, Andreas, et al. "Repaint: Inpainting using denoising diffusion probabilistic models." *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022.

[12] Kawar, Bahjat, et al. "Denoising diffusion restoration models." *Advances in Neural Information Processing Systems* 35 (2022): 23593-23606.
Rebuttal to rhkG (Part IV)
# Contribution Concerns

## Q4 Concept of ECI

We respectfully disagree with the reviewer's claim that ECI has been "*well studied*". We have noted in the previous clarification of the reviewer's misunderstanding that **extrapolation and interpolation have unique meanings in flow-based models** and are **completely different from concepts in regression-based methods** like the projected gradient. 

In flow matching, extrapolation and interpolation are geometric concepts related to the push-forward measures $\mu\_t=(\psi\_t)\_* \mu\_0$ on the probability path $\\{\mu_t,0\le t\le 1\\}$ [10] . Here $\mu_0$ is the initial (fixed) noise measure, $\psi_t$ is the flow (possibly defined with the learned vector field), and $(\psi_t)_*$ denotes the push-forward. When conditional on the target solution $u_1$​, the extrapolation along the conditional probability path can be written in a variational way in Equation 2. Similarly, interpolation are defined along the probability path. In this way, extrapolation is never a step to "minimize the objective", and interpolation has no correspondence with the projected gradient. Except for the correction stage which we do not claim as our contribution (see Q5), **we do not see any similarity between our ECI and the projected gradient, nor do we know any existing flow-based constrained generation model that uses a similar approach**. 

Furthermore, with an iterative sampling process, the intermediate generation is partially noises (see Figure 4), making the projected gradient infeasible, as it operates on the final prediction in regression models. This further distinguishes our framework from existing works.

## Q5 Novelty of ECI

The reviewer has a **significant misunderstanding of our contributions**. We do NOT claim any contribution in the correction stage  — we even manually placed it in the Appendix as it is less important. Instead, the **iterative control framework** with the concepts of extrapolation and interpolation unique to flow-based models (see Q4) using the interleaved ECI steps is our major claim for a better generation result. Furthermore, we almost **deliberately designed the correction step to be as simple as possible** to solidly demonstrate that the **good performance of the final generation comes from our iterative control framework** instead of from the correction algorithms like the projected gradient. 

Indeed, **even with such a naive correction algorithm, our ECI sampling still achieves remarkable generation results with hardly noticeable artifacts around the constrained region**. This is in sharp contrast with the ProbConserv baseline — a projection-based approach that directly operates on the final prediction without fine-grained iterative control — which exhibits noticeable and sharp change around the constrained region (see Figure 5 for a close-up result). Furthermore, in the correction stage in Figure 4, we demonstrate how the artifacts around the constrained boundary are gradually mitigated with our iterative ECI framework.

## Q6 Correction Step

We have already clarified in Q4 & Q5 that **the correction stage is never our contribution**. We also explain in Q5 why we deliberately designed the correction step as simple as possible. Regardless, ECI sampling still achieves state-of-the-art generation performance thanks to its **iterative control** over the flow sampling. New correction algorithms are always readily applicable to our framework but are not necessary. For experiments in our paper, we already demonstrated simple correction algorithms suffice.

## Q7 Hard-Constraint Generation

The hard-constraint generation task has **crucial importance** in physical systems like PDE, supply chains, and time series, where certain constraints like conservation laws have clear physical meanings and should always be respected. Previously, many regression models already tried to address this problem [1-4]. Our ECI sampling further extends such settings to generative modeling. Therefore, we respectfully disagree with the "*limited scope*" suggested by the reviewer, as we have already demonstrated the **effectiveness of ECI sampling across various 2D and 3D PDE datasets** and the potentially **wide applications in diverse SciML domains**.

We reiterate that extrapolation and interpolation have different and unique meanings in diffusion and flow-based generative models and that our contributions lie in the **iterative control of the flow sampling process by interleaving ECI steps**. This is completely different from any existing works, especially regression-based approaches like the projected gradient. In fact, though the extrapolation and interpolation for flow models (or the forward and backward diffusion processes for diffusion models) have been well-defined, **guiding diffusion or flow-based models is non-trivial**, and many existing works in the computer vision domain all adopted a similar zero-shot formulation [11, 12].
Rebuttal to rhkG (Part III)
# Theoretical Concerns

## Q1 Relation to projected gradient

We have already included a large amount of related work in the original manuscript. In the second part of related work, we reviewed existing work in the PDE domain for zero-shot constrained generation for **diffusion models**. In the third part, we also included related work in the computer vision domains which can be adapted for PDE tasks. We also include constrained prediction (*NOT generation*) work (e.g., [1-4]), which is essentially a special case of the projected gradient. However, we emphasize that we focus on **flow-based generative models**, so these regression approaches are **fundamentally different** from our problem step and can only serve as the correction stage of our framework.

We respectfully disagree with the reviewer's claim that "*many methods share many similarities with ECI in that they perform unconditional update steps, and then project back into the feasibility region*" and the lack of novelty. Though constrained prediction for regression models does exist (e.g., [1-4], as we have already cited them), they failed in the context of guiding diffusion or flow-based models that rely on iterative sampling. To the best of our knowledge, **we do not know any existing work for diffusion or flow-based models that share a similar idea with ours**. We would appreciate it if the reviewer could give more concrete references.



# Experimental Concerns

## Q2 Experimental Setup

We understand that the reviewer may be also not familiar with PDE domains. We noted all datasets and their spatiotemporal resolution setups are directly **taken from previous papers** (see Appendix B for reference of the datasets in previous papers). Therefore, we believe such an experimental setup is **standard, consistent, and comparable with existing works**. The datasets of PDE systems used in our work have also been the standard test cases in various papers (e.g. [5, 6]), which in turn demonstrated they are not "simple" cases, as the reviewer tried to suggest.

We further point out that the Navier-Stokes equation is a 3D dataset with 2D spatial and 1D temporal resolutions. To the best of our knowledge, **4D datasets in PDE systems do not exist so far**. It is also a **standard practice** to test on 2D and 3D datasets in the PDE domain, e.g., neural operator learning [5, 6], foundation models [7, 8], and existing constrained generation [2, 9]. Therefore, we believe we have already provided enough evidence of the superior performance of our proposed method.

We also believe the reviewer's claim that "*ECI performs poorly in complex settings*" is **ungrounded** due to their misinterpretation of our evaluation metrics (also see Q3). The MSE of the point-wise mean between the ground truth and the generation only partially reflects the resemblance of distributions. The MSE of the point-wise standard deviation is another metric, in which our model **significantly outperforms existing approaches**. Furthermore, inspired by the *Frechet inception distance* (FID), a widely used metric in image generation domains, we proposed to use the foundation model Poseidon to give a more comprehensive evaluation of the distributional property, in which our model excels over baselines. See Appendix D.1 for details.

## Q3 Variance in the solution

We have noted this is one of the **misinterpretations** regarding our evaluation metrics. We do not claim a higher variance is a sign of bad solutions, as the reviewer suggested. Instead, SMSE **measures the difference in the point-wise standard deviation** of the generation with respect to the ground truth solution set. Therefore, a smaller SMSE indicates **a closer resemblance of variance** to the ground truth solution distribution, thus a better generation result.
Rebuttal to rhkG (Part II)
Thank you for your appreciation of our work's **clear presentation**, **sufficient ablation studies**, and **effectiveness regarding hard constraints**. We will address your concerns as follows.

# Significant misunderstanding and misinterpretation of our work

We would first like to point out a **significant misunderstanding and misinterpretation** of our work. We address the constrained generation of the flow matching framework. The guided flow matching model during the iterative sampling process is still a **generative model** whose output is a **distribution over solutions** instead of a single ground truth solution. Sadly, we noted the reviewer emphasized the projected gradient approaches which are **infeasible for flow-based generative models**. The reviewer also **misinterpreted our generative evaluation metrics** and **emphasized constrained models in regression models** like projected gradient, which was not our claimed contribution.

We further distinguish our **problem and experimental setup** from existing work, especially for the projected gradient-related methods, in the following aspects.

- **Generative modeling**. Unlike projected gradient-related methods, our proposed framework is a generative framework that captures the distribution of solutions instead of a single solution.
- **Challenge**. The challenge for guiding flow (or diffusion) based models lies in its iterative sampling process, as we have mentioned in the second part of related work and Table 1. The iterative nature makes it infeasible to employ normal projected gradient approaches.
- **Gradient-free** guidance. Our proposed ECI-sampling is completely gradient-free. We demonstrated in our experimental results that our gradient-free approach still outperformed gradient-based methods while achieving significant speed-up.
- **Evaluation** metrics. The generative nature requires different evaluation metrics that capture distributional properties instead of simple MSE to the ground truth. In this regard, we calculate the point-wise MSE of the mean and standard deviation between the ground truth distribution and the generation distribution. We further apply Poseidon, a PDE foundation model to evaluate the Frechet distance.

We further reiterate our **contributions**, which we believe may have also been misinterpreted by the reviewer, as follows:

- We proposed ECI sampling for hard-constrained generation for pre-trained **flow-based generative models**. We emphasize *flow-based generative models*, which impose unique challenges due to their iterative sampling nature. Existing approaches for *constrained prediction* in regression models are infeasible or perform badly. To the best of our knowledge, we are the first to deal with the zero-shot flow-based constrained generation problems for functional data.
- **The extrapolation and interpolation concepts are unique to flow-based models**, which are completely different from any non-iterative approaches like project gradient. Indeed, we do not claim contributions in the correction — we even manually placed it in the Appendix as it is less important. Instead, the **iterative control framework** unique to flow-based models using the interleaved ECI steps is our major claim for a better generation result.
- Our approach guarantees the exact satisfaction of constraints **without the expensive gradient information**.
- Our approach achieves **state-of-the-art generative performance** on most evaluation metrics. We emphasize *generative evaluation metrics*, as the reviewer has drastically misinterpreted our metrics as regression MSEs. Instead, distributional properties are calculated and the foundation model Poseidon is used to give the most comprehensive metric.

We understand the reviewer might **not be familiar with generative modeling** in the PDE domain, which, as we have mentioned in the first part of our related work, is a newly emerged realm for functional data. However, as we have already pointed out the reviewer's misunderstanding and have distinguished our problem setup and contributions from existing work, we urge the reviewer to give a more fair and comprehensive evaluation of our work.
Rebuttal to rhkG (Part I)