RSW RMF RCLS RLLM
MT-Bench Prefix 100 ± 0 100 ± 0 100 ± 0 73 ± 5
Suffix 100 ± 0 100 ± 0 100 ± 0 84 ± 4
MMLU Prefix 90 ± 1 78 ± 4 100 ± 0 95 ± 1
Suffix 82 ± 2 63 ± 3 93 ± 1 93 ± 1
GSM8K Prefix 98 ± 0 100 ± 0 100 ± 0 100 ± 0
Suffix 94 ± 1 100 ± 0 100 ± 0 94 ± 3
Table 12: Average upgrade rates for different ways of adding the gadget to queries, in the white-box setting. Results are
similar in both methods, with a slight preference to the prefix approach.
RSW RMF RCLS RLLM
MT-Bench Uniform 100 ± 0 100 ± 0 100 ± 0 73 ± 5
Natural Prob. 100 ± 0 97 ± 2 100 ± 0 70 ± 5
MMLU Uniform 90 ± 1 78 ± 4 100 ± 0 95 ± 1
Natural Prob. 77 ± 2 41 ± 3 96 ± 2 87 ± 4
GSM8K Uniform 98 ± 0 100 ± 0 100 ± 0 94 ± 3
Natural Prob. 88 ± 2 92 ± 3 100 ± 0 83 ± 9
Table 13: Average upgrade rates for different ways of sampling candidate tokens during gadget generation, in the white-
box setting. Uniformly sampling the tokens yields better upgrade rates in most cases.
As mentioned in Section 5, to encourage the LLMs to follow the specific format in their responses (so they can be
parsed and compared with the ground-truth answers), we add a short prefix to the MMLU and GSM8K queries that
instructs the model how to respond. We phrase this instruction as follows: “ Answer the question using the format:
“Answer: [A/B/C/D]. Explanation: [EXPLANATION]” ” for the multi-choice queries of the MMLU benchmark, and a
similar version for GSM8K. We add this instruction after modifying the queries with the confounder gadget, i.e. the
instruction is prepended to the gadget.
An alternative to insert the instruction after the gadget but before the query, however we observed this to slighly underper-
form its counterpart. In the white-box setting we observe a slight decrease in the average (across all four routers) upgrade
rate from 91% to 89% for the MMLU benchmark, and from 98% to 91% for the GSM8K benchmark. In the black-box
setting, the average upgrade rate on MMLU reduces from 57% to 49% and on GSM8K from 73% to 64%.
Token sampling method. When generating the confounder gadget (see Section 4), we iteratively replace tokens with the
goal of maximizing the routing algorithm’s score for the gadget. Candidate replacement tokens are chosen uniformly at
random. An alternative is to choose candidates based on their probability of appearing in natural text. To evaluate this
method, we compute token probabilities by parsing and tokenizing the wikitext-103-raw-v1 dataset [44].
Table 13 shows that in most cases uniform sampling of replacement tokens yields better upgrade rates. We conjecture that
uniform sampling produces more unnatural text, confusing the router. For example, for the RSW routing algorithm, uni-
form sampling produces the following gadget: “legationbelongs967reglo’hui(DictionaryizedNameantal bidi.numberOf”,
whereas sampling according to natural probabilities produces “ total occurred According number Letar final Bab named
remainder”.
Number of tokens in the gadget. In our main evaluation, the gadgets are composed of n = 10 tokens. We evaluate the
effect of using less ( n = 5) or more ( n = 20 or n = 50) tokens. We observed that 5 tokens were insufficient to make
changes to the routing algorithm’s score and thus we were not able to optimize the gadget in this setting. As for 20 tokens,
we observe a a small improvement in the white-box setting, increase the average upgrade rate from 93.9% to 95.8%, and
a bigger improvement in the black-box setting, increase the average upgrade rate from 70.2% to 81.3%. Using 50 tokens
further increases the upgrade rates, to 98.2% in the white-box setting and 84.2% in the black box setting. The average
convergence rate increases as well, from 60 iterations for 10 tokens, to 70 for 20 tokens, and 100 for 50 tokens. Overall
this evaluation suggests that our rerouting attack can be even further improved by using longer gadgets, however it is