preliminary demonstration that the base model can
easily be fine-tuned to achieve good performance.
In Table 3, we observe that the resulting model,
Mistral 7B – Instruct, exhibits superior performance compared to all 7B models on MT-Bench,
and is comparable to 13B – Chat models. An
independent human evaluation was conducted on
https://llmboxing.com/leaderboard .
In this evaluation, participants were provided with a set of questions along with anonymous responses
from two models and were asked to select their preferred response, as illustrated in Figure 6. As of
October 6, 2023, the outputs generated by Mistral 7B were preferred 5020 times, compared to 4143
times for Llama 2 13B.
4
Figure 5: Results on MMLU, commonsense reasoning, world knowledge and reading comprehension for
Mistral 7B and Llama 2 (7B/13B/70B) . Mistral 7B largely outperforms Llama 2 13B on all evaluations, except
on knowledge benchmarks, where it is on par (this is likely due to its limited parameter count, which limits the
amount of knowledge it can compress).
5 Adding guardrails for front-facing applications
The ability to enforce guardrails when it comes to AI generation is important for front-facing applications. In this section, we highlight how to leverage system prompting to optionally enforce output