Be More Diverse than the Most Diverse: Online Selection of Diverse Mixtures of Generative Models
multi-armed bandits;evaluation of generative models;kernel-based evaluation scores
Yes
No ethics review needed.
In practice, FID metric is widely-used in the evaluation of generative models. Can this paper cover this metric and why?
1. Overall, this paper is well-written and easy to follow.
2. The proposed method (Mixture-UCB) is somehow novel, although it is inspired by classical UCB in the multi-armed bandit setting.
3. Theoretical results about the regret bound are provided for the proposed Mixture-UCB-CAB. The proof seems right although I have not checked the proof line-by-line.
4. Empirical results illustrate the effectiveness of the proposed method in finding the optimal mixture of text-based and image-based generative models.
This paper aims to solve the online selection task over a group of well-trained generation models. It explores the selection of a mixture of multiple generative models and formulate a quadratic optimization problem to optimize the kernel-based evaluation scores including kernel inception distance (KID) and Renyi kernel entropy (RKE). Specifically, it proposes an online learning approach called Mixture Upper Confidence Bound (Mixture-UCB). Theoretically, regret analysis is provided for one method (Mixture-UCB-CAB). Experimental results illustrate the effectiveness of the proposed method for text-based and image-based generative models.
1. I am afraid that the online selection of well-trained generative models might have few applications because it is already costly for the (large) generative model inference, then why do we need online selection rather than batch selection? Discussions about practical applications can be added.
2. Experimental results show that Mixture-UCB-OGD might be better than Mixture-UCB-CAB. However, theoretical guarantees about Mixture-UCB-OGD are missing. I know it might be more challenging and more detailed discussions can be added to clarify why.
Yes
No ethics review needed.
Below are a few questions and/or comments.

1. The problem appears novel, so I believe it makes sense to better motivative it. For example, in which context are we interested in picking a model to generate a sample at each round, why it is of interest to use "the fewest possible sample queries"? How the proposed method performs in an offline setting, with respect to performance and/or scalability?
2. When summarizing the contribution of this paper, could the authors also provide (forward) pointers to the precise results? For example, "proposing an online learning framework in Section ??". I personally believe that this may facilitate the interested readers to quickly grasp the main contribution of the paper.
3. Is the working assumption of linearly mixed model somewhat restrictive? Is there something else in the literature, or even such linear combination is (the first time) proposed by the authors in this paper? In fact, on the top row of Figure 3, there is a linearly mixtured "dog" that appears a bit bizarre: is this due to some limitation of this linear mixture? 
4. I personally find Theorem 1 a bit surprising: To me, kernel matrix "estimation" problem plus some online selection problem, and solving the former problem in general requires a lot of samples to have a tight spectral norm control on the estimated kernel matrix. I believe that the authors avoid this issue by assuming/focusing on the case of bounded. Could the authors comment more on this? For example, does this bounded kernel/loss function setting limit the practical interest of the proposed methods? Also, could the authors comment on the observed sample size $n_i$ for the proposed OGD method to make sense? We do not see this in Theorem 2 and this has an impact on the computational complexity I believe?
5. a tiny side remark: Figure 3 appears in the main text but commented in the appendix.
* The paper consider online selection of generative mixture models, which, to the best of knowledge, is a novel problem of interest.
* By making interesting connection to kernel-based scores and multi-armed bandit, the authors propose efficient methods to solve the above problem, with some theoretical guarantee. 
* Experiments on realistic data are provided, showing the practical applicability of the proposed approaches.
In this paper, the authors focus on the online selection of generative models, and in particular, the optimal linear mixture among a set of such models.
The problem appears novel, and the authors make interesting connections to the maximization of some kernel-based scores and multi-armed bandit. 
Based on this, the authors propose Algorithms 1 and 2 to solve this online selection of mixture efficiently, with performance guarantee given in Theorem 1 and 2, respectively. (Although I have some concerns on the settings and theoretical results, see below).
These methods can be used for widely used kernel inception distance (KID) and Renyi kernel entropy (RKE), and are tested on realistic image and text data in Section 6.
* It would be great to discuss the limitation of the proposed approach, see below for my detailed comments/questions.
* some settings and theoretical results need clarification, see below for my detailed comments/questions.
Yes
No ethics review needed.
* Can the authors find some other works that also aim to find good mixtures of generative models, and compare their method to these works?

* Can the authors provide the quality scores Density (Naeem et al., 2020). and Precision (Kynkaanniemi et al., 2019) in the experiments that they conducted?

* Small question regarding Lines 257&259: is $\hat{L}(\mathbf{a};\mathbf{x}^{(t)})-(\mathbf{\epsilon}^{(t)})^{\rm T}\mathbf{a})$ a lower or upper bound of $L(\mathbf{a})$?
* This paper is well written and easy to follow.

* The theoretical framework underlying the proposed algorithms is well grounded.

* Extensive experiments were carried out to demonstrate the performance of the proposed algorithms.
The main goal of this work is to maximize the diversity of generated samples by selecting not a single but a mixture of generative models. Formulating first a population loss of quadratic form that can translate into evaluation scores including kernel inception distance (KID) and Renyi kernel entropy (RKE), this article proposes two online algorithms based on continuum-armed bandit and gradient descent, to find the optimal mixture through minimizing an upper confidence bound of the quadratic population loss.  Experiments show that the proposed algorithms are efficient at approaching the optimal mixture.
* According to the literature review of this article, there seems to be little interest in finding a good mixture of different generative models. Indeed, if the goal is to approach the target distribution, it makes more sense to select the single best generative model than to use a mixture of different generative models, which are usually trained in an independent manner, therefore unlikely to complement each other.

* It is true that when the objective is to find the single best generative model, the online approach can help prevent sampling from suboptimal models. However, as using a mixture of generative models requires sampling from all member models, the online approach seems to be less useful in this setting.
Yes
No ethics review needed.
See above.
It's interesting to see the authors formulated the generative model selection problem as an online selection problem. The authors also developed two algorithms for this new setting and provide theoretical guarantees for one of them. Experimental results demonstrate the efficacy of the proposed algorithms.
This paper study online selection for generative models, in order to generate diverse samples. The authors formulated the problem as a mixture multi armed bandit problem and developed two algorithms for that: Mixture-UCB-CAB and Mixture-UCB-OGD. The authors developed theoretical guarantees for the Mixture-UCB-CAB algorithm. The authors conduct many experiments to show the efficacy of their developed methods.
1. Since this is a new problem, can authors provide more motivations for online selection of generative models, e.g., how important is the ability to generate diverse samples? And how important is to save samples in the selection process.
2. The authors provide a convergence guarantee for Mixture-UCB-CAB in Thm 2. For comparison, what is the rate of convergence for the offline approach that randomly generate $T$ samples and then optimize over $\alpha$?
3. Does Thm 1 holds for all $\alpha$? Also, the guarantee in Thm 2 doesn't suffer the curse of dimensionality even if the algorithm is selection $\alpha \in R^m$; can authors explain why does that happen?
4. Compared to standard bandit problem where one gets an intermediate regret term at each round, it seems that the studied problems gets $O(t)$ (averaged) terms (the first Eq in Section 5), and all these terms are related to the previous selections $x_1, \cdots, x_{t-1}$. Can authors elaborate how do they deal with these terms in the analysis? What are some technical contributions?
@inproceedings{
anonymous2024be,
title={Be More Diverse than the Most Diverse: Online Selection of Diverse Mixtures of Generative Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=2Chkk5Ye2s},
note={under review}
}
The availability of multiple training algorithms and architectures for generative models requires a selection mechanism to form a single model over a group of well-trained generation models. The selection task is commonly addressed by identifying the model that maximizes an evaluation score based on the diversity and quality of the generated data. However, such a best-model identification approach overlooks the possibility that a mixture of available models can outperform each individual model. In this work, we explore the selection of a mixture of multiple generative models and formulate a quadratic optimization problem to find an optimal mixture model achieving the maximum of kernel-based evaluation scores including kernel inception distance (KID) and Renyi kernel entropy (RKE). To identify the optimal mixture of the models using the fewest possible sample queries, we propose an online learning approach called *Mixture Upper Confidence Bound (Mixture-UCB)*. Specifically, our proposed online learning method can be extended to every convex quadratic function of the mixture weights, for which we prove a concentration bound to enable the application of the UCB approach. We prove a regret bound for the proposed Mixture-UCB algorithm and perform several numerical experiments to show the success of the proposed Mixture-UCB method in finding the optimal mixture of text-based and image-based generative models.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
multi-armed bandits evaluation of generative models kernel-based evaluation scores
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/2e5ac98ea7ebfabb4f0992b5c56b509717d1fe14.pdf
reinforcement learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/38e38b8b295e3069d30ca9af1d73c0dfad47a9d1.zip
Be More Diverse than the Most Diverse: Online Selection of Diverse Mixtures of Generative Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission