[71] L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. Xinget al., “Judging LLM-
as-a-judge with MT-Bench and chatbot arena,” Advances in Neural Information Processing Systems (NeurIPS) ,
2023.
[72] S. Zhu, R. Zhang, B. An, G. Wu, J. Barrow, Z. Wang, F. Huang, A. Nenkova, and T. Sun, “AutoDAN: Automatic
and interpretable adversarial attacks on large language models,” arXiv preprint arXiv:2310.15140, 2023.
[73] A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language
models,” arXiv preprint arXiv:2307.15043, 2023.
A Gadget Examples
Below are a few examples of the optimized confounder gadgets c we got, for the different routing algorithms:
• RSW :
legationbelongs967reglo’hui
(DictionaryizedNameantal bidi.numberOf
demanding RESULTS! XVI-ioibili Written Hormchriftlatable
• RMF :
041 personne.nama tomeTN Exact relac ¸˜ao tinha seis.Description
NUMAGEMENTMedian157 mut-camerauacao nije Hir Ter
• RCLS :
contiene DEM USERNAME fourteen chees781 consultants200 inici DOJ
571:
Ord:nth Norwegian Mercer docs Abr226 METADATA
• RLLM :
dated:frameifyumi345 Kurdasciiuzeiphertext
Midnightexecution431!784 below1 unwrap : / n / n
B Ablation Study
In this section, we evaluate the effect of different hyperparameters and design choices (in the white-box setting).
Prefix vs. suffix. As described in Section 4, we prepend the confounder gadget to the query. An alternative is to append
it. This is straightforward for MT-bench and GSM8K, but MMLU consists of multi-choice questions followed by a list
of possible answers, and the term “Answer:”. We insert the gadget at the end of the question text and before the possible
answers. If we append it at the very end, after “Answer:”, the LLM assumes the query was answered and in many cases
does not generate any output at all.
Table 12 shows that average upgrade rates are similar regardless of whether the gadget was inserted as a prefix or a suffix.
For MMLU, prefix works better. The downgrade rate is 0% in all cases.
22