text: LLM routers. A prominent example of this emerging class of LLM control planes are LLM routers [27, 41, 47, 53, 59]. 
  LLM routers decide which of the two (or, sometimes, more) LLMs to use to answer a query. In prescriptive routing, 
  the router applies some lightweight classifier to the input query that determines which underlying LLM to utilize for a 
  response. The classifier is itself a learned function that scores the complexity of the query. Deployments can then configure 
  a score threshold for when to route a query to the more expensive LLM. This threshold can be tuned using representative 
  workloads to achieve a desired cost-performance trade-off. Figure 1 shows the basic workflow of binary LLM routers. 
  
  Non-prescriptive routing [15, 20, 68] uses the responses from one or more underlying LLMs to determine which response 
  to return to the user. For example, FrugalGPT [20] submits the query to a sequence of models (ordered by price) called a 
  cascade, stopping when it obtains a response classified by the router as sufficient. 
  
  2 