Non-prescriptive routing [15, 20, 68] uses the responses from one or more underlying LLMs to determine which response
to return to the user. For example, FrugalGPT [20] submits the query to a sequence of models (ordered by price) called a
cascade, stopping when it obtains a response classified by the router as sufficient.
2