0 50 100 150 200 250 300
Perplexity
0
20
40
60
80Count
Original
Confounded
(a) RSW
20 40 60 80 100 120 140
Perplexity
0
10
20
30
40
50Count
Original
Confounded (b) RMF
50 100 150 200
Perplexity
0
10
20
30
40
50Count
Original
Confounded (c) RCLS
20 40 60 80 100
Perplexity
0
10
20
30
40
50Count
Original
Confounded (d) RLLM
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.5
0.6
0.7
0.8
0.9
1.0TPR
AUC = 1.00
(e) RSW
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.4
0.5
0.6
0.7
0.8
0.9
1.0TPR
AUC = 0.97 (f) RMF
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.4
0.5
0.6
0.7
0.8
0.9
1.0TPR
AUC = 0.99 (g) RCLS
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0TPR
AUC = 0.95 (h) RLLM
Figure 5: Perplexity of the original queries in the GSM8K benchmark compared to the perplexity of confounded queries
using a single uniformly sampled gadget. We additionally present the ROC curve of the defense that detects confounded
queries by checking if they cross a perplexity threshold, and it’s corresponding ROCAUC score. Confounded queries have
significantly higher perplexity values, and are thus easy to recognize and filter out.
where PPL(·) denotes the perplexity function computed using GPT-2, the value ρ denotes a target perplexity value to
which we want gadgets’ perplexity to be close, and the value α is a balancing coefficient. For the experiments below, we
set ρ to be the average perplexity value of 100 uniformly sampled queries3 from the GSM8K benchmark.
Figure 6 shows the results when setting α = 0.01, for the GSM8K benchmark and one confounder gadget. The results
demonstrate that modified queries can no longer be easily distinguished from normal queries by their perplexity alone.
For instance, in the case of the RSW router, setting the threshold value at 55 as before, no confounded queries are flagged
as anomalous, meaning the true-positive rate is zero. We note that there is some variability across gadgets. The average
ROCAUC scores of the defense across ten gadgets with standard deviation indicated parenthetically, are 0.66 (±0.04),
0.69 (±0.02), 0.71 (±0.02), and 0.69 (±0.03) for the RSW , RMF , RCLS , and RLLM routers, respectively.
At the same time, optimizing for low perplexity does not significantly impact the attack success rate. Table 11 compares
the average upgrade rates (over n = 10 gadgets) of the original perplexity-agnostic optimization approach from Section 4
and the perplexity-minimizing one described above. The attack efficacy might be improvable further by adjusting α to
find a sweet spot that avoids the defense effectively while ensuring high rerouting success rate.
The attack is not particularly sensitive to the choice of queries used to obtain the calibration value ρ. Although ρ was
computed using GSM8K queries, we observe similar performance when evaluating on the MT-bench and MMLU bench-
marks, with average ROCAUC scores of0.50 (±0.01), 0.51 (±0.01), 0.52 (±0), and 0.51 (±0.01) for MT-bench, and0.52
(±0.03), 0.54 (±0.02), 0.55 (±0.01), and 0.53 (±0.02) for MMLU. One might also try removing the calibration value al-
together, instead simply minimizing the gadget’s perplexity value. However, this can result with an “overshooting” effect,
where the perplexity value is significantly lower than that of normal queries, thereby making it still distinguishable from
standard queries.
In summary, perplexity-based filtering is not an effective defense against against rerouting.
3The perplexity calibration queries were chosen such that they do not overlap with the queries used for evaluation.
15