From Conflicts to Convergence: A Zeroth-order Method for Multi-Objective Learning
Zeroth-order method，Multi-objective learning，Optimization，Generalization
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
@misc{
liu2024from,
title={From Conflicts to Convergence: A Zeroth-order Method for Multi-Objective Learning},
author={Xinyue Liu and Xinzhe Yuan and Hong Chen and Bin Gu and Huan Xiong and Zhihao Li and Xuelin Zhang},
year={2024},
url={https://openreview.net/forum?id=RsDYaswfEj}
}
Multi-objective learning (MOL) is a popular paradigm for learning problems under multiple criteria, where various dynamic weighting algorithms (e.g., MGDA and MODO) have been formulated to find an updated direction for avoiding conflicts among objectives. Recently, increasing endeavors have struggled to tackle the black-box MOL when the gradient information of objectives is unavailable or difficult to be attained.  Albeit the impressive success of zeroth-order method for single-objective black-box learning,  the corresponding MOL algorithm and theoretical understanding are largely absent. Unlike single-objective problems, the errors of MOL introduced by zeroth-order gradients can simultaneously affect both the gradient estimation and the gradient coefficients $\lambda$, leading to further error amplification. To address this issue, we propose a Stochastic Zeroth-order Multiple Objective Descent algorithm (SZMOD), which leverages function evaluations to approximate gradients and develops a new decomposition strategy to handle the complicated black-box multi-objective optimization. Theoretically, we provide convergence and generalization guarantees for SZMOD in both general non-convex and strongly convex settings. Our results demonstrate that the proposed SZMOD enjoys a promising generalization bound of   $\mathcal{O}(n^{-\frac{1}{2}})$, which is comparable to the existing results of first-order methods requiring additional gradient information. Experimental results validate our theoretical analysis.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Xinyue_Liu3 ~Xinzhe_Yuan1 ~Hong_Chen1 ~Bin_Gu1 ~Huan_Xiong1 ~Zhihao_Li12 ~Xuelin_Zhang3
Xinyue Liu Xinzhe Yuan Hong Chen Bin Gu Huan Xiong Zhihao Li Xuelin Zhang
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Zeroth-order method，Multi-objective learning，Optimization，Generalization
I certify that there is no acknowledgement section in this submission for double blind review.
liu|from_conflicts_to_convergence_a_zerothorder_method_for_multiobjective_learning
/pdf/6b8621de9f4522bb6d8f583b9d60d0e0cd5841ac.pdf
optimization
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/44d3e6b1cebdf27bfdf4946f1c39cc1c7ea62180.zip
From Conflicts to Convergence: A Zeroth-order Method for Multi-Objective Learning
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission