in implementing new features and integrating their solutions into ours. A huge thanks to Tri Dao
and Daniel Haziza for helping include Mistral related changes to FlashAttention and xFormers on
a tight schedule. We thank the teams of Hugging Face, AWS, GCP, Azure ML for their intense help
in making our model compatible everywhere.
6
Figure 6: Human evaluation of Mistral 7B – Instruct vs Llama 2 13B – Chat Example. An example of
human evaluation from llmboxing.com . The question asks for recommendations of books in quantum physics.
Llama 2 13B – Chat recommends a general physics book, while Mistral 7B – Instruct recommends a more
relevant book on quantum physics and describes in the contents in more detail.
7
References
[1]Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and
Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head
checkpoints. arXiv preprint arXiv:2305.13245 , 2023.
[2]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David
Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large