20 30 40 50
Perplexity
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0Count
Original
Confounded
(a) RSW
20 30 40 50
Perplexity
0
5
10
15
20Count
Original
Confounded (b) RMF
20 30 40 50
Perplexity
0
5
10
15
20Count
Original
Confounded (c) RCLS
20 30 40 50
Perplexity
0
5
10
15
20Count
Original
Confounded (d) RLLM
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.0
0.2
0.4
0.6
0.8
1.0TPR
AUC = 0.65
(e) RSW
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.0
0.2
0.4
0.6
0.8
1.0TPR
AUC = 0.73 (f) RMF
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.0
0.2
0.4
0.6
0.8
1.0TPR
AUC = 0.64 (g) RCLS
0.0 0.2 0.4 0.6 0.8 1.0
FPR
0.0
0.2
0.4
0.6
0.8
1.0TPR
AUC = 0.65 (h) RLLM
Figure 6: Perplexity values of the original and confounded queries, and the corresponding ROC curves of the defense that
detects confounded queries by checking if they cross a perplexity threshold, when the confounder gadget is optimized for
low perplexity, in the GSM8K benchmark and for one gadget sampled uniformly at random. Confounded queries have
similar perplexity values as the original queries, and can no longer be easily distinguished based on perplexity alone.
RSW RMF RCLS RLLM
Orig. PPL-opt. Orig. PPL-opt. Orig. PPL-opt. Orig. PPL-opt.
MT-Bench 100 ± 0 100 ± 0 100 ± 0 98 ± 2 100 ± 0 98 ± 1 73 ± 5 51 ± 8
MMLU 90 ± 1 59 ± 5 78 ± 4 74 ± 5 100 ± 0 66 ± 12 95 ± 1 89 ± 3
GSM8K 98 ± 0 70 ± 7 100 ± 0 98 ± 2 100 ± 0 88 ± 6 94 ± 3 81 ± 8
Table 11: Average upgrade rates for gadgets generated without (“Orig.”) and with (“PPL-opt.”) low-perplexity optimiza-
tion, for the balancing coefficient α = 0.01. In some cases, optimizing for low perplexity has a negative effect on the
attack success rate, however the attack can still be considered successful. A more careful choice ofα can potentially limit
the effect on the attack success.
LLM-based filtering. Even though adversarially modified queries cannot be easily detected using perplexity, they may
still be “unnatural.” A possible defense is to employ an oracle LLM to determine if the query is natural or not. This defense
requires the router to invoke an additional LLM for every processed query, which is computationally expensive in the case
of a high-quality open-sourced LLM or financially costly in the case of a high-quality commercial LLM. Therefore, this
defense is unlikely to be practical. Furthermore, it is possible to optimize gadgets so that they both have low perplexity
and appear “natural” to LLM evaluators [69].
Paraphrasing. Filtering defenses like those discussed above are passive. An active alternative is to paraphrase queries
using an oracle LLM. LLMs are trained to generate natural text and are thus likely to remove unnatural substrings when
paraphrasing a query. This defense is likely impractical for two reasons. First, and as with LLM-based filtering, it requires
16