Easing Training Process of Rectified Flow Models Via Lengthening Inter-Path Distance
Rectified Flow Models;Training;Easing;Distance-Aware Noise-Sample Matching;DANSM
Yes
No ethics review needed.
Q1- I would appreciate a better explanation for Figure 6 (b), the plot looks a bit unclear to me. Additionally, there are lines behind the legend curbing my ability to understand the plot completely. My essential question is that unlike Figure 7, in Figure 6 (b) I do not see any gains in training times when using the proposed method. Is this understanding of mine correct?
The work is well written except for a few typos (for example, in line 198/199 "To **analysis**" is written instead of "To **analyse**", and in line 309/310 "have not been **evaluation** on RFM" is written instead of **evaluated**).

The idea is sound and is theoretically motivated, the results are in line with the hypothesis.
The implementation seems simple and straightforward. 

The claims made in the paper seem to be backed by empirical evaluations.
This work proposes Distance-AwareNoise-Sample Matching (DANSM) to increase training speed in generative models without any loss (in fact gain) in performance. The proposed method is inspired by "the difference between random paths used in training and preferable paths from well-trained models".
The work is theoretically motivated and sound.
The work offers limited evaluations, however, they seem to be sufficient to make the case.
I have 2 major points in this regard:
1. The plots are not well made, some of the lines in the plots, for example, Figure 6(b), Figure 7(b), and Figure (b) seem to be starting from arbitrary locations. It would help to invest more time and make better-looking plots without these artifacts, or if they are not artifacts then an explanation for the same would be very helpful.

2. The work lacks comparisons to "Immiscible Diffusion", while the paper does point out the key differences in lines 307-315, it would be interesting to see empirical evaluations in comparison to "Immiscible Diffusion" since the methods are closely related.
Yes
No ethics review needed.
1. Include a comparison using longer sampling steps.

2. Conduct more extensive experiments on complex datasets.
1. The diagram is very clear, making it easy to understand how different diffusion methods yield similar results with the same dataset, identifying "preferable noises" for samples.

2. The proposed method significantly speeds up training by 30%–40% without losing quality, offering new insights into enhancing diffusion model efficiency.
This paper explores how different diffusion methods yield similar results with the same dataset, identifying "preferable noises" for samples. It introduces the Distance-Aware Noise-Sample Matching (DANSM) method to optimize training by increasing inter-path distances. DANSM significantly speeds up training by 30%–40% without losing quality, offering insights into enhancing diffusion model efficiency.
1. The FID calculations are measured with very few sampling steps, making it difficult to ensure the quality of the generated results. It would be beneficial to provide a comparison of different methods while maintaining image quality, ideally with visualizations, such as images decoded using stable diffusion.

2. The experiments were only validated on CIFAR and LSUN-BEDROOM datasets. Validation on a broader range of datasets, such as ImageNet and FFHQ, would provide more comprehensive insights. Additionally, including some actual generated images for visual comparison would be advantageous.
Yes
No ethics review needed.
Why is it necessary to learn latent-noise pairs that are well-aligned in the t-SNE space? This might suggest that straight flow matching is preferable, but the underlying reasoning is not clearly explained in the paper. Furthermore, I am confused about whether "consistent model reproducibility" is beneficial or detrimental for diffusion-based models. The fact that different model architectures can generate similar images from the same noise might indicate limited generation patterns due to the isotropic nature of the Gaussian noise.
The paper presents a framework that simplifies the optimization of inter-path distances to path length optimization, based on the observations of noise-sample pair relationships in diffusion models.

The approach leverages closed-form formulas derived from rectified flow models to enable efficient optimization without requiring architectural modifications, making it easy to integrate with existing methods.
The paper reveals that different diffusion models trained on the same dataset tend to produce similar outputs when given the same input noise. This suggests the existence of "preferable noise-sample pairs" in the training process. The authors propose the Distance-Aware Noise-Sample Matching method to lengthen the inter-path distance for speeding up the training of diffusion-based models. The experiments show that the proposed method improves the training speed by about 30%~40%.
The authors heavily emphasize "consistent model reproducibility," which seems interesting but abruptly transitions to proposing a method for improving training speed. This makes the main idea of the paper quite unclear.

The presentation of the proposed method is vague, and Algorithm 1 cannot be effectively reproduced with the provided steps. I suggest adding mathematical formulations in Sections 3.3 and 4.2 to rigorously explain how the proposed model is optimized.

Regarding experiments, the paper lacks visual comparison results, which are crucial for method evaluation, especially when the FID quantitative metrics show minimal differences at the tiny scale discussed. I also recommend comparing with more methods and incorporating the proposed training speed improvement approach across more SD models. I believe the paper should present higher-resolution and larger-scale generation results. Given the computational benefits brought by this work's methodology, these tasks should be more computationally feasible.
easing the training process of rectified flow models by better noise-sample pairs in each training epoch
@inproceedings{
anonymous2024easing,
title={Easing Training Process of Rectified Flow Models Via Lengthening Inter-Path Distance},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RaR3ETzyKp},
note={under review}
}
Recent research pinpoints that different diffusion methods and architectures 
trained on the same dataset produce similar results for the same input noise. 
This property suggests that they have some preferable noises for a given sample. 
By visualizing the noise-sample pairs of rectified flow models and stable diffusion models in two-dimensional spaces, 
we observe that the preferable paths, connecting preferable noises to the corresponding samples, 
are much well organized with significant fewer crossings comparing with 
the random paths, connecting random noises to training samples. 
In high-dimensional space, paths rarely intersect. 
The path crossings in two-dimensional spaces indicate the shorter inter-path distance 
in the corresponding high-dimensional spaces. 
Inspired by this observation, we propose the Distance-Aware Noise-Sample Matching (DANSM) method 
to lengthen the inter-path distance for speeding up the model training. 
DANSM is derived from rectified flow models, which allow using a closed-form formula to calculate the inter-path distance. 
To further simplify the optimization, we derive the relationship between inter-path distance and path length, 
and use the latter in the optimization surrogate. 
DANSM is evaluated on both image and latent spaces by rectified flow models and diffusion models. 
The experimental results show that DANSM can significantly improve the training speed by 30\% $\sim$ 40\%
without sacrificing the generation quality.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Rectified Flow Models Training Easing Distance-Aware Noise-Sample Matching DANSM
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/3aa5f5ea64cd2d3cb2d969a3dacf4a5790aecbc1.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Easing Training Process of Rectified Flow Models Via Lengthening Inter-Path Distance
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission