and reasoning benchmarks.
4Since Llama 2 34B was not open-sourced, we report results for Llama 1 34B.
3
Figure 4: Performance of Mistral 7B and different Llama models on a wide range of benchmarks . All
models were re-evaluated on all metrics with our evaluation pipeline for accurate comparison. Mistral 7B
significantly outperforms Llama 2 7B and Llama 2 13B on all benchmarks. It is also vastly superior to Llama 1
34B in mathematics, code generation, and reasoning benchmarks.
Model Modality MMLU HellaSwag WinoG PIQA Arc-e Arc-c NQ TriviaQA HumanEval MBPP MATH GSM8K
LLaMA 2 7B Pretrained 44.4% 77.1% 69.5% 77.9% 68.7% 43.2% 24.7% 63.8% 11.6% 26.1% 3.9% 16.0%
LLaMA 2 13B Pretrained 55.6% 80.7% 72.9% 80.8% 75.2% 48.8% 29.0% 69.6% 18.9% 35.4% 6.0% 34.3%