References
[1] “Chatbot Arena LLM Leaderboard: Community-driven evaluation for best LLM and AI chatbots,” https://
huggingface.co/spaces/lmarena-ai/chatbot-arena-leaderboard, accessed: 2024-11-14.
[2] “Hello gpt-4o,” https://openai.com/index/hello-gpt-4o/, published: 2024-05-23.
[3] “Introducing Llama 3.1: Our most capable models to date,” https://ai.meta.com/blog/meta-llama-3-1/, published:
2024-07-23.
[4] “Introducing Meta Llama 3: The most capable openly available LLM to date,” https://ai.meta.com/blog/
meta-llama-3/, published: 2024-04-18.
[5] “Martian LLM router,” https://withmartian.com/.
[6] “New embedding models and API updates,” https://openai.com/index/new-embedding-models-and-api-updates,
published: 2024-01-25.
[7] “Notdiamond LLM router,” https://www.notdiamond.ai/.
[8] “OpenAI and others seek new path to smarter AI as current meth-
ods hit limitations,” https://www.reuters.com/technology/artificial-intelligence/
openai-rivals-seek-new-path-smarter-ai-current-methods-hit-limitations-2024-11-11, published: 2024-11-15.
[9] “OpenAI, Google and Anthropic are struggling to build more advanced AI,” https://www.bloomberg.com/news/
articles/2024-11-13/openai-google-and-anthropic-are-struggling-to-build-more-advanced-ai?sref=CrGXSfHu,
published: 2024-11-13.
[10] “OpenAI shifts strategy as rate of ‘GPT’ AI improvements slows,” https://www.theinformation.com/articles/
openai-shifts-strategy-as-rate-of-gpt-ai-improvements-slows, published: 2024-11-9.
[11] “Openrouter LLM router,” https://openrouter.ai/.
[12] “Unify LLM router,” https://unify.ai/.
[13] “What is a control plane?” https://www.ibm.com/think/topics/control-plane, published: 2024-10-31.
[14] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman,
S. Anadkat et al., “GPT-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
[15] P. Aggarwal, A. Madaan, A. Anand, S. P. Potharaju, S. Mishra, P. Zhou, A. Gupta, D. Rajagopal, K. Kappaganthu,
Y . Yanget al., “Automix: Automatically mixing language models,” arXiv preprint arXiv:2310.12963, 2023.
[16] G. Alon and M. Kamfonas, “Detecting language model attacks with perplexity,” arXiv preprint arXiv:2308.14132,
2023.
[17] R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. the method of paired comparisons,”
Biometrika, vol. 39, no. 3/4, 1952.
[18] N. Carlini, D. Paleka, K. D. Dvijotham, T. Steinke, J. Hayase, A. F. Cooper, K. Lee, M. Jagielski, M. Nasr, A. Conmy
et al., “Stealing part of a production language model,” arXiv preprint arXiv:2403.06634, 2024.
[19] H. Chaudhari, G. Severi, J. Abascal, M. Jagielski, C. A. Choquette-Choo, M. Nasr, C. Nita-Rotaru, and A. Oprea,
“Phantom: General trigger attacks on retrieval augmented language generation,” arXiv preprint arXiv:2405.20485,
2024.
[20] L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving
performance,” arXiv preprint arXiv:2305.05176, 2023.
[21] W.-L. Chiang, L. Zheng, Y . Sheng, A. N. Angelopoulos, T. Li, D. Li, B. Zhu, H. Zhang, M. Jordan, J. E. Gon-
zalez, and I. Stoica, “Chatbot arena: An open platform for evaluating LLMs by human preference,” in Forty-first
International Conference on Machine Learning (ICML), 2024.
[22] S. Cho, S. Jeong, J. Seo, T. Hwang, and J. C. Park, “Typos that broke the RAG’s back: Genetic attack on RAG
pipeline by simulating documents in the wild via low-level perturbations,”arXiv preprint arXiv:2404.13948, 2024.
[23] J. Chu, Y . Liu, Z. Yang, X. Shen, M. Backes, and Y . Zhang, “Comprehensive assessment of jailbreak attacks against
LLMs,” arXiv preprint arXiv:2402.05668, 2024.
[24] K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakanoet al.,
“Training verifiers to solve math word problems,”arXiv preprint arXiv:2110.14168, 2021.
[25] N. Dalvi, P. Domingos, Mausam, S. Sanghai, and D. Verma, “Adversarial classification,” inProceedings of the tenth