text: As mentioned in Section 5, to encourage the LLMs to follow the specific format in their responses (so they can be 
  parsed and compared with the ground-truth answers), we add a short prefix to the MMLU and GSM8K queries that 
  instructs the model how to respond. We phrase this instruction as follows: “Answer the question using the format: 
  “Answer: [A/B/C/D]. Explanation: [EXPLANATION]”” for the multi-choice queries of the MMLU benchmark, and a 
  similar version for GSM8K. We add this instruction after modifying the queries with the confounder gadget, i.e. 
  the 
  instruction is prepended to the gadget. 
  
  An alternative to insert the instruction after the gadget but before the query, however we observed this to slighly underper- 
  form its counterpart. In the white-box setting we observe a slight decrease in the average (across all four routers) upgrade 
  rate from 91% to 89% for the MMLU benchmark, and from 98% to 91% for the GSM8K benchmark. In the black-box 
  setting, the average upgrade rate on MMLU reduces from 57% to 49% and on GSM8K from 73% to 64%. 