Maximum Likelihood Estimation for Flow Matching by Direct Second-order Trace Objective
flow matching;generative models
Yes
No ethics review needed.
N/A
This paper provides a method for directly optimizing the high-order objective, extending the scope of previous work.
This paper provides a method for directly optimizing the high-order objective for flow matching.
1. The theoretical results are not novel.
2. The improvement is marginal.
Yes
No ethics review needed.
See weekness.
1.The paper introduces a novel approach that directly minimizes high-order objectives, rather than just their upper bounds, thereby theoretically reducing KL divergence more effectively.

2.The proposed method demonstrates better performance in terms of likelihood estimation (Negative Log-Likelihood) and 2-Wasserstein distance in experiments on both 2D synthetic and high-dimensional image datasets.

3.The use of Hutchinson's trace estimation method reduces the computational cost of calculating high-order objectives, making the approach more efficient.
The paper proposes a method for directly minimizing high-order objectives in Maximum Likelihood Estimation (MLE) for flow matching models. The proposed method directly minimizes the higher-order objectives, leading to improved likelihood estimation. The effectiveness of this approach is demonstrated through experiments on 2D synthetic datasets and high-dimensional image datasets, showing superior likelihood and data generation quality compared to previous works.
1.While the paper claims that directly minimizing high-order objectives leads to better likelihood maximization, it does not provide rigorous formal guarantees or convergence proofs that this method will always outperform minimizing upper bounds in all scenarios. The theoretical analysis is somewhat limited in terms of offering strong mathematical assurances for the proposed approach's superiority.

2.The theoretical results rely on several assumptions, such as bounding the Fisher divergence by a function of high-order objectives. These assumptions may not always hold in practical applications, especially when dealing with more complex data distributions or models.

3.The experiments are incomplete: the methods compared in the paper have not been evaluated on all datasets. The method proposed in this paper did not achieve state-of-the-art (SOTA) results on CIFAR-10 and ImageNet, and on MNIST, it was only compared with one method.
Yes
No ethics review needed.
refer to the question above
1.The proposed method minimizes the KL divergence by directly addressing the second-order objective, offering a more reliable optimization of likelihood compared to earlier methods.
2.The approach demonstrates robustness across various types of datasets, including 2D synthetic and high-dimensional image datasets, indicating its scalability and versatility.
The authors propose directly minimizing a high-order objective, specifically the second-order objective, to overcome the limitations of previous methods that focused only on minimizing upper bounds. This new approach aims to optimize likelihood more effectively by reducing the KL divergence between the data distribution and the generated distribution. Experimental results show that the proposed method performs better than existing flow matching techniques on both 2D synthetic datasets and high-dimensional image datasets. So,
to summary, this paper has 
1.Introduced a direct minimization technique for high-order objectives in flow matching, which enhances maximum likelihood estimation.
2.Utilized the gradient of the conditional vector field to calculate the second-order objective without needing simulations, improving computational efficiency.
3.Provided empirical evidence that the proposed method leads to better likelihood compared to previous approaches on several datasets.
1.The method requires explicit computation of the second-order objective, which can be computationally intensive for very high-dimensional datasets, potentially limiting its applicability to extremely large-scale cases.
2.More details about experimental settings, such as learning rate and number of training epochs, need to be provided.
3.The tables in the paper have a lot of empty space below them, which affects the overall formatting. The layout of the paper should be reorganized for better presentation.
We proposed a method to maximize likelihood more directly in flow matching.
@inproceedings{
anonymous2024maximum,
title={Maximum Likelihood Estimation for Flow Matching by Direct Second-order Trace Objective},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=7ZUUNMjM9T},
note={under review}
}
Flow matching, one of the attractive deep generative models, has recently been used in wide modality.
Despite the remarkable success, the flow matching objective of the vector field is insufficient for maximum likelihood estimation. 
Previous works show that adding the vector field's high-order gradient objectives further improves likelihood.
However, their method only minimizes the upper bound of the high-order objectives, hence it is not guaranteed that the objectives themselves are indeed minimized, resulting in likelihood maximization becoming less effective.
In this paper, we propose a method to directly minimize the high-order objective.
Since our method guarantees that the objective is indeed minimized, our method is expected to improve likelihood compared to previous works.
We verify that our proposed method achieves better likelihood in practice through experiments on 2D synthetic datasets and high-dimensional image datasets.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
flow matching generative models
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/cad7d06032d8d22e7d1a928d4534b013f1f8015d.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Maximum Likelihood Estimation for Flow Matching by Direct Second-order Trace Objective
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission