0 10 20 30 40 50 60 70
Perplexity
0
5
10
15
20Count
strong
weak
(a) MT-bench
ROCAUC=0.38
0 20 40 60
Perplexity
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0Count
strong
weak
(b) MMLU
ROCAUC=0.47
0 20 40 60 80
Perplexity
0
5
10
15
20
25Count
strong
weak
(c) GSM8K
ROCAUC=0.38
Figure 7: Histograms of the perplexity values of clean responses generated by the strong and weak models.
RSW RMF RCLS RLLM
Up. Down. Up. Down. Up. Down. Up. Down.
MT-Bench 0 ± 0 24 ± 2 0 ± 0 67 ± 6 0 ± 0 29 ± 3 24 ± 3 1 ± 0
MMLU 8 ± 3 9 ± 2 0 ± 0 77 ± 7 0 ± 0 50 ± 4 55 ± 4 5 ± 1
GSM8K 4 ± 2 48 ± 9 1 ± 1 78 ± 11 0 ± 0 80 ± 4 21 ± 4 4 ± 2
Table 16: Upgrade and downgrade rates for the downgrading variant of our rerouting attack, where the goal is to reroute
queries to the weak model (white-box).
As mentioned in Section 5, throughout our evaluations we filter out responses with perplexity values higher than 100.
This is due to a few responses getting arbitrarily high perplexity values although corresponding to valid responses. For
example, for the query:
Suppose you are a mathematician and poet. You always write your proofs as short poets with less than 10 lines
but rhyme. Prove the square root of 2 is irrational number.
The weak model responses with:
In squares, two seeks a home, Two whole numbers, never roam. If it did, in pairs combined, A different square
would it find. But take root, two’s square, or four, Still leaves a remainder’s roar. The square root of two’s no child
of two, Its irrational nature ever true.
which results with an high perplexity value of 166. We also observed a few responses containing lists, code, or math
equations with abnormally high perplexity values.
E Rerouting to the Weak Model
In this section we evaluate the generality of our attack and show that generation of confounder gadgets can be optimized
for the opposite objective from what we consider so far: reroute queries to the weak model. For this, we repeat the same
optimization process as in Section 4 but minimize the router’s score. Table 16 shows the upgrade and downgrade rates
for this variant of the attack, in the white-box setting. In most cases we see a significant downgrade rate and a minimal
upgrade rate, meaning that most of the modified queries were routed to the weak model. One notable exception is the
LLM-based router RLLM , for which the attack does not work well. Future work will be needed to explore improving
confounder generation for this setting further.
25