Training Robust Ensembles Requires Rethinking Lipschitz Continuity
robustness;lipschitzness;ensembles;convolutions;transferability;black-box attacks
Yes
No ethics review needed.
I addressed the questions in the Weaknesses section.
Overall I'm a fan of establishing (through published literature) empirical observations of interesting deep learning phenomena. This is an example of such a thing. The fact that we can both a) make an individual model more robust via Lipschitz constant changes, but at the same time b) make an ensemble of such models more similar in a way that reduces the overall robustness of the ensemble, is a very curious piece of empirical evidence.

I think the evaluation is pretty strong and despite some obvious additions (that I mention later) I think it is sufficient to establish this as a phenomenon that might be real and that should be investigated further.
This paper makes 3 contributions to the field of robust ensemble training: 

1. It identifies a trade-off in neural network ensembles: while decreasing the Lipschitz constant of individual models improves their robustness, it unfortunately also increases the transferability of adversarial examples between models in the ensemble, potentially undermining the ensemble's overall robustness. 

2. It introduces LOTOS (Layer-wise Orthogonalization for Training rObust enSembles), a new training method that promotes orthogonality between corresponding layers of different models in an ensemble. LOTOS is computationally efficient and pretty architecturally agnostic. 

3. Through experiments on CIFAR-10 and CIFAR-100, the authors demonstrate that LOTOS improves ensemble robustness against black-box attacks by 6 percentage points, and when combined with existing robust ensemble methods, enhances their performance by up to 10.7 percentage points.
There are a few weakness that I believe could be relatively easily addressed.

### Limitation 1: Testing only against a single type of adversarial attack
As far as I understand it, the authors only test the robustness against one type of an adversarial attack -- the PGD-50 = Projected Gradient Descent with 50 iterations. I think it is important to 1) add more types of attacks (including black box ones), and 2) possibly look at some standard attack suites such as RobustBench (https://github.com/RobustBench/robustbench)

### Limitation 2: Adversarial robustness vs other kinds of robustness
What about other kinds of robustness such as robustness to noise, sheer, etc. Do you see similar phenomena to what you describe under adversarial attacks?

### Limitation 3: Attack strength norms and their effect on the phenomenon and LOTOS
I would be very curious what the effect of the attack norm (L_infty vs L_2 etc) and attack size (8/255, ...) is on the observed phenomenon and the efficacy of LOTOS. I can imagine that different attack size norms might lead to different qualitative tradeoffs. Could you please be more specific with what exactly you do and how does this relate to your theory & experiments?

### Limitation 4: Weak models?
The results and improvements you present are (I think) pretty non-robust to begin with. Am I wrong about this? How do robust model perform under the same attack? If the model you are using does not have a high robustness to begin with, I believe this might lead to misleading results on what helps and what doesn't. This is because many things tend to help when we're not close to the maximum robustness, but might not generalize. **Addressing this is crucial for me to raise your score.**
Yes
No ethics review needed.
- Could you compare your methods with other ensemble defenses, especially with DVERGE?
- Could you consider stronger attacks such as AutoAttack [b] and MORA [a] which specifically target ensemble models?

[a]: Yu et al., MORA: Improving Ensemble Robustness Evaluation with Model-Reweighing Attack. NeurIPS 2022. https://arxiv.org/abs/2211.08008
[b]: Croce et al., Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML 2020. https://arxiv.org/abs/2003.01690
This paper links decreasing Lipschitz continuity with increasing transferrability. This is an interesting point of exploration.
This paper finds that while Lipschitz continuity improves individual model robustness, it unexpectedly raises adversarial transferability in model ensembles. To counter this, the authors introduce LOTOS to boost ensemble robustness by promoting orthogonality between models.
- What are the white-box attacks that you used in this paper? Some attacks may not be reliable in assessing ensemble robustness.
- The method appears to be an improvement upon TRS. However, TRS was shown to have an overestimated robustness [a], and it is not well-explained why the LOTOS loss is an improvement.
- It is not clear to me why DVERGE was not considered in the baseline comparisons, as it is known to be an effective robust baseline for ensemble defenses.
- Section 4 is poorly written. I am having trouble understanding your method. For eq. (3):
  - Where is $w_i$ introduced?
  - Why does this equation measure similarity?
  - What are $A$ and $B$ matrices, are they model parameters?
  - Also given that $A$ and $B$ are trainable, do you backprop through the SVD of them?
- For eq. (4): There is no $k$ in the LHS.
- Theorem 4.1:
  - "In Theorem 4.1, we prove even k = 1 can be effective in orthogonalization with respect to the remaining singular vectors for convolutional layers." It is not clear to me how it is proven for $k = 1$.
  - Why circular padding?
  - It is not clear to me how this theorem links with eq. (4).
Yes
No ethics review needed.
- The experiments in the paper focuses on ResNet and DLA. Do you think the method will extend to other popular architectures such as vision transformer, which operates differently and can be shallower?
- Line 215 (using the difference in the population loss of the two models on these adversarial examples as a proxy) -- How good is this approximation? If we use some alternative loss functions other than cross-entropy, is there any chance for this approximation become exact?
- In Equation (3), how would we determine $w_i$? Why do we need this weight, because intuitively we could perhaps just use the singular values here?
- Line 260 ($\\| \sum_{i=1}^d \sigma_i u_i v_i^\top v_i' \\|_2 = 0$). Here we only try to make the singular vectors orthogonal for each $i$, but the singular vectors for different $i$'s are not accounted. Would it possible to have the case where, for example, the first singular vector of $f^{(j)}$ is orthogonal to the first singular vector of $g^{(j)}$, but highly similar to the second singular vector of $g^{(j)}$? In this case, is the transferability between $f$ and $g$ still low?
The paper analyzes an interesting problem, and uses the conclusions to propose a method that seems to work.
The paper analyzes the relationship between model Lipschitzness and the transferability of adversarial examples. Under the setting of TRS model ensemble, decreasing the model's Lipschitz constant improves standalone robustness but also increases transferability, which hurts ensemble robustness. To this end, the authors propose "LOTOS", which promotes the orthogonality of the singular vectors corresponding to the largest singular values of the transformation matrices of the layers of the ensemble candidate models. LOTOS can decrease transferability while only having a slight effect on individual robustness, thereby improving the ensemble robustness. Furthermore, LOTOS can work with heterogeneous model architectures in conjunction with adversarial training.
First of all, I would like to disclose that I am not an expert on black-box attacks or diversity-promoting robust ensembles. I am more familiar with the literature about the white-box scenario.

- The proposed method seems promising in the black-box setting, but how about the white-box case? I think this question is important because it will disentangle robustness from attack difficulty.
- What would happen if the "source model" in the black-box transfer attack is a LOTOS model itself (trained separately from the target model)?
- All definitions in this paper seem to build on the $\ell_2$ norm. Is the analyses and the proposed method only applicable to $\ell_2$ attacks? How about other attack budgets?
- Just to clarify, are all models in the ensemble trained simultaneously, or can they be trained sequentially? Asking for clarification here because the proposed loss function for each model depends on the weights of all other models.
- Line 285 says "for the orthogonalization to be effective in Equation (4), it is necessary to increase the value of $k$ (dimension of orthogonal sub-spaces)". Then, Line 294 says "$k = 1$ can be effective in orthogonalization with respect to the remaining singular vectors for convolutional layers." Are these not contradictory?
- In Theorem 4, the right-hand-side of Equation 5 has two terms under the square root. Which one is dominant? $\epsilon^2$ or the other terms?
- Line 333: typo "ensebmels".
Yes
No ethics review needed.
- What is the definition of $\Re$ in the appendix (page 14)?
- Typos: line 333: "ensebmels"
- Do you think the method in this paper requires significant work on hyperparameter tuning?
- It's interesting to observe the trade-off between the robustness of individual models and the transferability of adversarial examples when changing Lipschitz constant. It highlights the importance of addressing the trade-off, an issue that has been overlooked in prior research.
- The proposed LOTOS method is presented as a solution that uniquely focuses on orthogonalizing affine layers, setting it apart from existing methods that target different aspects of model diversity. To my best knowledge, such contribution is also novel.
- The empirical results are shown clearly and shed light on so much understanding of the proposed solution.
The paper studies how Lipschitz continuity affects the transferability of adversarial examples in model ensembles. It finds that while lower Lipschitz constants improve individual model robustness, they increase transferability in ensembles. To address this, the authors propose LOTOS, a training method that promotes subspace orthogonality between models, enhancing ensemble robustness. Experiments show that LOTOS improves the robust accuracy of ResNet-18 ensembles and boosts the performance of existing robust ensemble methods.
- **The soundness of the claim in this paper**: The authors suggest that "although a lower Lipschitz constant increases the robustness of a single model, it is not as beneficial in training robust ensembles as it increases the transferability rate of adversarial examples across models in the ensemble." While I believe that the first part is sound, I want to challenge the statement "a lower Lipschitz constant is not as beneficial in training robust ensembles as it increases the transferability rate of adversarial examples across models in the ensemble". There are too many aspects that may influence the result, for instance, the generalization of the models (because a lower Lipschitz constant may make the models more difficult to train). In other words, on a broad view, it's not just a trade-off between two aspects mentioned in this paper. I encourage the authors to incorporate deeper understanding of this issue (see below).
- **Missing related work and discussion**: As far as I know, there may be some papers that is relevant to yours. For example, some theoretical works [1,2,3] discuss the smoothness & Lipschitz constant in transferability of adversarial examples (see the bounds therein), which may be somehow relevant to Lipschitz continuity emphasized in this paper. Some of them may not discuss the model ensemble issue. However, these insights on individual models is theoretically connected with the analysis in this paper. Furthermore, the vulnerability-diversity decomposition [4] also sheds light on the relationship among transferability, single model robustness and diversity, which may be in line with your claim "We observe that while decreasing the Lipschitz constant makes each model of the ensemble individually more robust, it makes them less diverse and consequently increases the transferability rate among them which in turn hinders the overall ensemble robustness". In addition, the model ensemble literature [5] also suggests the trade-off between diversity and individual model performance. Therefore, maybe you could give a brief discussion of them to help us better understand your theoretical insights. You are also encouraged to choose to discuss other work that I haven't mentioned but is relevant to your paper.
- **Experiments**: Based on the above, I encourage the authors to incoporate more experiments. Since the model generalization / optimization process may also change the trade-off mentioned in this paper, you may add more experiments regarding more algorithms to show the soundness and consistency of your claim. However, in the current version, the experimental parts only consider their own method as well as TRS [1].

If the author's answers address my concerns, I will consider raising the score.

[1] Yang Z, Li L, Xu X, et al. Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness[J]. Advances in Neural Information Processing Systems, 2021, 34: 17642-17655.

[2] Zhang Y, Hu S, Zhang L Y, et al. Why does little robustness help? a further step towards understanding adversarial transferability[C]//2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024: 3365-3384.

[3] Wang Y, Farnia F. On the role of generalization in transferability of adversarial examples[C]//Uncertainty in Artificial Intelligence. PMLR, 2023: 2259-2270.

[4] Yao W, Zhang Z, Tang H, et al. Understanding Model Ensemble in Transferable Adversarial Attack[J]. arXiv preprint arXiv:2410.06851, 2024.

[5] Abe T, Buchanan E K, Pleiss G, et al. Pathologies of predictive diversity in deep ensembles[J]. arXiv preprint arXiv:2302.00704, 2023.
We show that although a lower Lipschitz constant increases the robustness of a single model, it is not beneficial in training robust ensembles. We introduce LOTOS, a new training paradigm for ensembles, which counteracts this adverse effect.
@inproceedings{
anonymous2024training,
title={Training Robust Ensembles Requires Rethinking Lipschitz Continuity},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=WKW5TG8ItY},
note={under review}
}
Transferability of adversarial examples is a well-known property that endangers all classification models, even those that are only accessible through black-box queries. Prior work has shown that an ensemble of models is more resilient to transferability: the probability that an adversarial example is effective against most models of the ensemble is low. Thus, most ongoing research focuses on improving ensemble diversity. Another line of prior work has shown that Lipschitz continuity of the models can make models more robust since it limits how a model's output changes with small input perturbations. {\em In this paper, we study the effect of Lipschitz continuity on transferability rates.} We show that although a lower Lipschitz constant increases the robustness of a single model, it is not as beneficial in training robust ensembles as it increases the transferability rate of adversarial examples across models in the ensemble. Therefore, we introduce LOTOS, a new training paradigm for ensembles, which counteracts this adverse effect. It does so by promoting orthogonality among the top-$k$ sub-spaces of the transformations of the corresponding affine layers of any pair of models in the ensemble. We theoretically show that $k$ does not need to be large for convolutional layers, which makes the computational overhead negligible. Through various experiments, we show LOTOS increases the robust accuracy of ensembles of ResNet-18 models by $6$ percentage points (p.p) against black-box attacks on CIFAR-10. It is also capable of combining with the robustness of prior state-of-the-art methods for training robust ensembles to enhance their robust accuracy by $10.7$ p.p.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
robustness lipschitzness ensembles convolutions transferability black-box attacks
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/69bc850d477ab17f36ee30ea9614bc1eb97812e3.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/48f7ecdb4d491a80f593ee9d750a916a4395189d.zip
Training Robust Ensembles Requires Rethinking Lipschitz Continuity
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission