an extra potentially expensive LLM invocation for each query processed by the router. Second, it may degrade the quality
of responses from the destination LLMs, which are sensitive to the phrasing of queries and prompts.
Detecting anomalous user workloads. Another possible defense requires the router to monitor individual user work-
loads, and identify those users whose queries are routed to the strongest model with an abnormally high frequency. The
router can then impose a user-specific threshold. Of course such workloads may have a benign explanation, e.g., the user’s
queries may be unusually complex. Even so, routers could potentially be designed to perform user-specific routing. For
example, one could imagine using per-user thresholds that are calibrated dynamically to attempt to maintain a consistent
fraction of queries being routed to the strong model.
Such user-specific routing would complicate implementations, and would make inaccurate decisions for a user until there
is sufficient data about their queries. The latter is relevant in adversarial settings, since such an approach would still be
circumventable should attackers be able to mount Sybil attacks in which the attacker creates a new user for, in the limit,
each query.
9 Related Work
Evasion attacks against ML systems. A large body of work has investigated evasion attacks against ML systems [25,
43, 60], also referred to as adversarial examples [32, 48, 49], and these attacks are now being explored in the context of
multi-modal LLMs [28] as well as text-only LLMs (for just one example, see [22]). We discussed in Section 3 how our
results compare: LLM control plane integrity is a distinct AI safety issue, but related in that: (1) control plane integrity
attacks may use evasion-style techniques, and (2) control plane integrity attacks might be useful for performing evasion.
Prompt injection against LLMs. Prompt injection is a class of attacks against LLMs in which the adversary manipulates
the prompt, i.e., the textual input fed directly to the LLM, causing the LLM to generate outputs that satisfy some adver-
sarial objective [50, 64]. Evasion attacks as discussed above can use prompt injection, jailbreaking attacks being a widely
explored example in which the adversary aims to bypass some safety guardrail included in the LLM system, such as “do
not output expletives” [23, 42, 54, 66, 72, 73].
Prompt injection is also used for extraction attacks that aim to infer some information from or about the model, for
example, the system prompt [50, 54, 70], training data samples [46], or model parameters [18]. In indirect prompt injection
attacks [33], the adversaries do not directly interact with the target LLM, and instead inject adversarial inputs into third-
party data, which is then added to the LLM prompt (intentionally or unintentionally) by the victim application and/or its
users. This relates to another category of attacks that target LLM-based applications, such as RAG systems, and invalidate
their integrity by exploiting the weaknesses of the underlying LLM [19, 55].
Our attacks also modify queries, but with a different aim than the above types of attacks: undermining the integrity of the
control plane routing, rather than the LLM itself. Future work might investigate indirect control plane integrity attacks
that, analogously to indirect prompt injection, serve to somehow trick users of a routing system into forming control-
plane-confounding queries.
Attacks against MoE. Mixture-of-Experts (MoE) architectures enable using multiple expert modules for processing a
given query with a lower computational cost by including an inner routing mechanism that in every layer routes different
tokens to a small number of experts [29, 30, 52, 56]. This can be thought of as an internal router within a single LLM,
rather than an external control plane that orchestrates multiple LLMs. MoE has increased in popularity as it allows to