Learning to Optimize for Mixed-Integer Nonlinear Programming
Mixed-Integer Nonlinear Programming;Learning to Optimize;Differentiable Optimization;Constrained Neural Networks;Deep Learning;Operations Research
Yes
No ethics review needed.
Could you please expand on the constrain violations produced by your method? Do you have an understanding of how the proposed method handle different constraint functions? For instance, what type of constraint are well-handled? What, instead, are more difficult to satisfy?
The paper is well written and organized. The idea of integrating the rounding operations within model training is sound and allows to obtain superior models with respect to Learning to Optimize models who solve a relaxed version and perform the rounding operations at inference time, as shown in the experimental sections. Computational advantages are also significant with respect to traditional numerical solver.
The paper proposes an end-to-end method for learning solutions of integers programs by enabling differentiation through the rounding operation within model training. This is done by using the Straight-through Estimator (STE) combined with the Gumbel-noise method, which smooths the discrete function representing the rounding operations to obtain useful gradients for backpropagation. The paper provides a comprehensive evaluations of the proposed method across several optimization tasks.
My main concern is that the proposed method cannot ensure constraint satisfactions, since it uses a soft constraint approach. I believe that integer variables also makes difficult to perform projections to restore feasibility at inference time. Nonetheless, the percentage of infeasible solution generated by the proposed method is low, and the results shown in the Table 5 suggest that using a Lagrangian-inspired method might allow to obtain a better estimate of the dual variables, which might help to reduce constraint violations. 
The paper might benefit from a more systematic evaluation of the impact of different constraint functions to the feasibility/violations produced by the proposed method, which might allow to identify scenarios and pattern where the proposed method (does not) produce constraint violation.
Yes
No ethics review needed.
- Could you please explain how the proposed method handles problems with different sizes?
- Could the proposed method generalize to large instances, such as those with thousands of constraints and variables?
- The topic of MINLP is an interesting and important topic in the field of learning to optimize.
- This paper combines the gradient information during the optimization.
- The presentation in this paper is good.
This paper proposes two differential correction layers (rounding classification and learnable threshold) that generate integer outputs while preserving gradient information.  The experiments demonstrate that the proposed learning-based approach consistently produces high-quality solutions.
- The STE is not novel in the ML field. Moreover, the author may want to explain that combining the gradient information cannot lead to local optima.
- While many works on learning to optimize use GNN to process problems with different sizes, the proposed method seems to use MLP with fixed-size inputs. Thus, the network may fail to process problems of various sizes.
- The author may investigate the effects of different $\lambda$ on the performance.
- The author may conduct experiments on more complex instances, and the 60-second time limit is too short. Existing works in learning to optimize conduct experiments on challenging instances with at least 1000 sec of time limit [1,2].

[1]  A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming

[2] GNN&GBDT-Guided Fast Optimizing Framework for Large-scale Integer Programming
Yes
No ethics review needed.
1. In lines 330 to 334, the authors mention modifications made to the original quadratic problems from Donti et al. (2021). However, it remains unclear whether these modifications provide any advantages to the proposed method. A clarification on this point is necessary.

2. In Algorithm 1, the authors only consider a round-down direction for integer variables. It would be beneficial to explain why the round-up direction is excluded. If the round-up direction is relevant, this should be described in detail.

3. In the experiments, the authors allocate only a 60-second time budget to the exact solver. This limited timeframe may hinder the solver’s ability to find the optimal feasible solution, even if a few additional seconds are provided. It would be more informative to present a statistical distribution of % Infeasible versus Time (seconds) for the various methods evaluated.
1. MINLPs arise in numerous real-world applications, making the techniques proposed in this paper significantly relevant to practical problem-solving.

2. The paper is well-structured and clearly articulated, making it accessible to the reader.

3. The authors assert that they are the first to introduce Straight-Through Estimator (STE) and Gumbel-Sigmoid techniques in the context of learning-to-optimize, which they identify as pivotal for efficiently generating solutions to large-scale MINLP problems.
This paper addresses the challenging problem of Mixed-Integer Nonlinear Programming (MINLP) within a learning-to-optimize framework, a crucial area of research with significant applications across various domains. The integration of learning approaches into MINLPs is particularly complex due to the presence of integer decision variables, which complicates gradient-based optimization techniques. To tackle this issue, the authors propose two differentiable correction methods that enable neural networks to generate high-quality integer solutions while maintaining gradient information for backpropagation. Additionally, the authors conduct a comprehensive set of experiments to demonstrate the superiority of their proposed methods compared to traditional exact search algorithms and heuristic approaches.
1. The fairness of the comparisons and the definitions used in the experiments are unconvincing for several reasons:

   - In lines 324 to 327, the authors list the solvers compared and the corresponding types of problems. However, they do not provide sufficient justification for the selection of these solvers or explain their relevance to the specific problem types addressed.

   - In lines 330 to 334, the authors mention modifications made to the original quadratic problems from Donti et al. (2021), but it remains unclear whether these modifications confer any advantages to the proposed method. Clarification is needed.

   - The metrics employed in the experiments raise concerns. For instance, while generating low percentages of infeasible solutions quickly is noted, the implications of this metric are questionable. The time required to convert an infeasible solution into a feasible one can be substantial, thus diminishing the significance of the reported speed.

   - In the experiments involving simple nonconvex problems, the use of the %Unsolved metric is unconventional. It is problematic to claim a problem is solved when the provided solution is still infeasible.

2. The loss function introduced in the paper essentially applies the Lagrangian multiplier method, which is not particularly novel in this field.

3. Additionally, there are several typographical errors throughout the paper. The authors should conduct a thorough proofreading before submission.
Yes
No ethics review needed.
See the weakness part.
This paper focuses on applying machine learning methods to solve MINLPs. 
- It proposes novel differentiable correction layers that can potentially handle the non-differentiability of integer outputs in deep learning models.
This paper proposes an end-to-end optimization method for solving general mixed-integer nonlinear programs (MINLPs). The proposed approach consists of two steps to generate solutions. In the first step, a neural network is employed to generate a relaxed solution that is close to the optimal solution. In the second step, another neural network provides update directions for continuous variables and rounding rules for integer variables. All of these neural networks are trained in a self-supervised manner. The Straight-Through Estimator is utilized to manage non-differentiable operations, such as rounding.
I have a few serious concerns below.

- First and foremost, since the proposed approach does not take advantage of the non-linear part, I think it could also be applicable to mixed-integer linear programs. Then why not also conduct computational experiments on those instances and show how good or bad it performs? We know that learning to solve MINLPs is rarely studied but being the first to address such a problem could be a trivial thing (not a significant contribution). 

- Note that in the computational studies, only the right hand sides of constraints are perturbed, I recommend the authors perturb all parameters in the MINLP formulations and conduct experiments. The reason I ask such a question is, representing MINLPs using neural networks itself is a very important question (and challenging). Note that representing linear programs or mixed-integer linear programs via neural networks has theoretical foundations, see [1] [2]. Furthermore, I do not see equality constraints in the dataset. 

- Can the authors consider more practical MINLP instances? Such as MINLPLIB (https://www.minlplib.org/). The dataset used in the manuscript is kind of like toy problems. I'm expecting to see the computational performances on real-life instances.

- The parameter $\lambda$ in loss function is an import hyper-parameter for balancing feasibility and optimality, and should be analyzed more carefully. Usually, penalty methods in L2O demonstrate very weak generalization capabilities. This kind of explains why the infeasibility ratio in Table 4 is so high. I do not think penalizing constraints in the loss function is a good way. Rather, the authors should design special algorithms to handle nonlinear (and possibly non-convex) constraints. 

[1] Chen, Z., Liu, J., Wang, X., Lu, J. and Yin, W., 2022. On representing linear programs by graph neural networks. arXiv preprint arXiv:2209.12288.

[2] Chen, Z., Chen, X., Liu, J., Wang, X. and Yin, W., 2024. Expressive Power of Graph Neural Networks for (Mixed-Integer) Quadratic Programs. arXiv preprint arXiv:2406.05938.
@inproceedings{
anonymous2024learning,
title={Learning to Optimize for Mixed-Integer Nonlinear Programming},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=1oIXRWK2WO},
note={under review}
}
Mixed-integer nonlinear programs (MINLPs) arise in various domains, such as energy systems and transportation, but are notoriously difficult to solve. Recent advances in machine learning have achieved remarkable success in optimization tasks, an area known as learning to optimize. This approach includes using predictive models to generate solutions for optimization problems with continuous decision variables, thereby avoiding the need for computationally expensive optimization algorithms. However, applying learning to MINLPs remains challenging primarily due to integer decision variables, which complicate gradient-based learning. To address this limitation, we propose two differentiable correction layers that generate integer outputs while preserving gradient information. The experiments demonstrate that the proposed learning-based approach consistently produces high-quality solutions for parametric MINLPs extremely quickly. As problem size increases, traditional exact solvers and heuristic methods struggle to find feasible solutions, whereas our approach continues to deliver reliable results. Our work extends the scope of learning-to-optimize to MINLP, paving the way for integrating integer constraints into deep learning models.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Mixed-Integer Nonlinear Programming Learning to Optimize Differentiable Optimization Constrained Neural Networks Deep Learning Operations Research
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/0f1b7892bdc29a1c970846e717a71f74bcdb5a24.pdf
optimization
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Learning to Optimize for Mixed-Integer Nonlinear Programming
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission