COFlowNet: Conservative Constraints on Flows Enable High-Quality Candidate Generation
generative flow network;offline RL;molecule design
Yes
No ethics review needed.
**Questions 1:** Regarding the weakness in literature, is there any other task to demonstrate the efficiency of the proposed method? If so, it would be much better to add one or two more results of COFlowNet.
**Strength 1:** The main strength of this work is the broad impact on any candidate-generation tasks.
- I feel that the method can be applied to any candidate-generation task, such as molecule design. Also, as pointed out in the paper, flow generation can be further used to search plausible flows to find the optimal decision-making in the RL study.
This paper tackles the generation task, which aims to search for state transitions with high rewards. The fundamental task strongly impacts generating or searching for plausible candidates of sequences, where its possible applications include molecule design or path generation.
To this end, the authors have proposed COFlowNet, which focuses on finding high-reward state transitions by simply regularizing unsupported flows. The efficacy of the method is confirmed in Hypergrid and Molecule Design tasks.
**Weakness 1:** The comparison with prior works and the evaluations seem limited.
- By referring the baseline paper of GFlowNet, it is compared with other non-flow-based approaches such as MCMC (MARS) and PPO. However, COFlowNet is only compared with GFlowNet, which makes it hard to figure out the quantitative gains of COFlowNet over MCMC and PPO. It is better to add these two baselines in the experiments.
- Moreover, only the main experiment considered in this paper, which includes the comparisons with others, is Molecule Design. (I have not published works for this topic) In literature, is there any other task to demonstrate the efficiency of the proposed method? A single demonstration seems to be limited to say the consistent and meaningful gains of COFlowNet over others.

**Minor Comment 1:** For consistency, please choose the one among COFlowNet or COFlownet.
Yes
N/A
No ethics review needed.
1. No references are provided for vanilla GFlowNet and QM-GFlowNet. There is no justification that these are the state-of-the-art methods.
2. Although COFlowNet w/o shows that the regularization term enhances performance, but can not beat the competing method QM-GFlowNet. The experiment results are weak. If the experiment results can not be improved, please provide a more in-depth analysis of where COFlowNet falls short compared to QM-GFlowNet.
3. Are there other potential ways of increasing diversity during GFlowNet regularization that were explored aside from the quantile matching algorithm? 
4. Can COFlowNet be easily extended to other domains where RL is popular, like game playing and autonomous vehicle driving for example?
The authors demonstrate originality in their work by clearly outlining their research problems in the context of other popular works and systematically laying out solutions for them with corresponding theorems and proofs. While maintaining an easy-to-read and high-level dialogue in addressing their approach’s significance, aim, and scope, the authors also manage to spare no mathematical details when necessary. Theorem 1 and the writing in sections 3.3 and 4 show formally the details required to both implement and understand the proposed approach.
The paper present a method to the problem of offline GFlowNet training. Many conventional approaches require approximating training scores/rewards of candidate model outputs to provide a variety of generated candidate solutions, however, this approach is sensitive to out-of-distribution data. In this work the authors remove any reliance on proxy models for estimating the candidate to reward optimization objective for GFlowNet training. The authors propose to constrain the actions that a GFlowNet model may take to generate an output sample such that only actions supported by the training data are permissible. The authors show that their flow regularization only positively affects training on the supported flow paths, and that their approach is truly offline and can improve candidate model diversity.
In theorem 2, the use of the phrase “be more close to policy of offline dataset” seems too imprecise to be in the theorem statement. Since the proof is rather short this could perhaps be moved to the end of the proof, so that reader’s still get the high-level point that the author is trying to convey. Additionally, the conclusion of the proof for theorem 2 seems to be absent, although the technical/algebraic details seem sound. 

Although the authors effectively demonstrate the efficacy of COFlowNet on the Molecule Design problem, providing experimental results on datasets from other domains would help show the success of their method beyond a shadow of a doubt.
Yes
No ethics review needed.
See weakness
+ Adapting GFlowNets for offline scenarios is a valuable idea.
+ The proposed method shows superiority over baseline models like QM-GFlowNet and FM-GFlowNet.
In this paper, the author introduces COFlowNet, an offline Generative Flow Network (GFlowNet) designed to generate high-quality and diverse candidates without relying on online evaluations. The effectiveness of COFlowNet is validated through experiments on commonly used datasets.
1. The authors conduct experiments on Hypergrid and molecule design tasks. Could the proposed method be generalized to more tasks and real applications? It is recommended to conduct experiments on additional tasks.

2. How much time and computational cost are saved by training in the offline setting? Could the authors compare the training cost between the proposed method and traditional methods?

3. It would be better to use \textbf{} for the first sentence in figure and table captions to indicate their targets clearly.

4. It would be  better to indicate the meaning of 'mixed', 'expert', and 'random' in Figure4.

5. Do number of modes and ℓ1 error are commonly used in evaluation of GFLowNets? Are there more metrics to measure the performance?
We propose an offline version of generative flow network.
@inproceedings{
anonymous2024coflownet,
title={{COF}lowNet: Conservative Constraints on Flows Enable High-Quality Candidate Generation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=tXUkT709OJ},
note={under review}
}
Generative flow networks (GFlowNet) have been considered as powerful tools for generating candidates with desired property. Given that evaluating the property of candidates can be complex and time-consuming, existing GFlowNets train proxy models for efficient online evaluation. However, the performance of proxy models is heavily dependent on the amount of data and is of considerable uncertainty. Therefore, it is of great interest that how to develop an offline GFlowNet which does not rely on online evaluating. 
Under offline setting, the limited data results in insufficient exploration of state space. 
The insufficient exploration means that offline GFlowNets can hardly generate satisfying candidates out of the distribution of training data. Therefore, it is critical to restrict the offline model to act in distribution of training data. The distinctive training goal of GFlownets poses unique challenge for making such restriction.
Tackling the challenge, we proposes Conservative Offline GFlowNet (COFlowNet) in this paper. We define unsupported flow, edges containing unseen states in training data. Models can learn extremely few knowledge about unsupported flow from training data. By constraining the model from exploring unsupported flows, we restrict COFlowNet to explore as optimal trajectories on the training set as possible, thus generating better candidates. In order to improve the diversity of candidates, we further introduce quantile version of unsupported flow restriction. Experimental result on several widely-used datasets validates the effectiveness of COFlowNet on generating high-scored and diverse candidates. All implementations are available at \href{https://anonymous.4open.science/r/COflownet-2872}{https://anonymous.4open.science/r/COflownet-2872}
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
generative flow network offline RL molecule design
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ec95c39fe9e77624947469a328adb032629e6ffa.pdf
reinforcement learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
COFlowNet: Conservative Constraints on Flows Enable High-Quality Candidate Generation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission